Table 2 1–3–5 Evaluation Indicator set.
First-class Indicator | Second-class Indicator | Third-class Indicator |
|---|---|---|
I-1 User experience performance | II-1 Usefulness | III-1 Timeliness of the medical information provided by AI-DSS |
III-2 accuracy of the medical information provided by AI-DSS | ||
III-3 usability of the medical information provided by AI-DSS | ||
II-2 Convenience | III-4 Operability of AI-DSS | |
III-5 Reliability of AI-DSS | ||
III-6 Compatibility of AI-DSS with HIS | ||
III-7 Entire response speed of AI-DSS | ||
II-3 Accessibility | III-8 Implementation rate of AI-DSS | |
III-9 Training frequency for AI-DSS utilization | ||
III-10 Average resolution time for bug fixes or requests fulfilled during AI-DSS utilization | ||
II-4 Privacy | III-11 Ratio of alerts generated in AI-DSS data acquisition | |
III-12 Accounting for patient individuality, beliefs, and preferences in AI-DSS utilization | ||
II-5 Satisfaction | III-13 Satisfaction with AI-DSS functionality | |
III-14 Satisfaction with AI-DSS flexibility | ||
III-15 Satisfaction with AI-DSS effectiveness | ||
III-16 Satisfaction with AI-DSS | ||
III-17 Satisfaction with AI-DSS safety | ||
I-2 Organizational performance | II-6 Cost | III-18 Implementation cost of AI-DSS |
III-19 Operational maintenance cost of AI-DSS | ||
III-20 Training cost for AI-DSS utilization | ||
II-7 Efficiency | III-21 Average consultation duration for patient in the clinic | |
III-22 Average time taken for physician decision-making in the clinic | ||
II-8 Workability | III-23 alignment ratio of diagnoses between healthcare professionals and AI-DSS | |
III-24 False negative diagnosis rate in AI-DSS | ||
III-25 Ratio of alerts generated for high-risk diseases in AI-DSS | ||
III-26 Volume of clinical decision support generated by AI-DSS | ||
II-9 Risk | III-27 Proportion of AI-DSS undergoing ethical review pre-deployment | |
III-28 Algorithm transparency of AI-DSS | ||
III-29 Algorithm interpretability of AI-DSS | ||
III-30 Risk level in integrating AI-DSS with hospital information systems | ||
II-10 Foreground | III-31 Enhancement of hospital reputation through AI-DSS | |
III-32 Optimization budget for AI-DSS | ||
I-3 Societal performance | II-11 Innovation | III-33 Extent of process reengineering for improved healthcare services via AI-DSS |
III-34 System update frequency for AI-DSS | ||
II-12 Effect | III-35 Average patient waiting time | |
III-36 Healthcare professionals' compliance with standards and guidelines | ||
III-37 Patient satisfaction with healthcare services | ||
III-38 Average cost for each outpatient visit | ||
II-13 Feasibility | III-39 Extent of relevant policy support at the national level | |
III-40 Extent of relevant policy support at the local level | ||
III-41 Extent of hospital informatization | ||
II-14 Security | III-42 Traceability of data acquisition process | |
III-43 Documentation of AI-DSS modifications in Patient Medical Data | ||
III-44 Incidence Rate of Patient Medical Data Breaches | ||
III-45 Occurrence of Medical Injuries Linked to AI-DSS | ||
II-15 Sustainability | III-46 Trustworthiness of AI-DSS | |
III-47 Acceptance of AI-DSS |
