Table 1 Clinical Trials Framework for Artificial Intelligence Applications
Phase and Purpose | Outcomes | Overview | Steps |
|---|---|---|---|
Phase 1: Safety | Model performance | Pre-deployment planning and preparation | - Engage diverse stakeholders - Compare training and target populations - Develop strategies to prioritize underserved populations - Design initial workflows - Initial bias/fairness analyses using retrospective data - Clinical utility, workflow integration |
Phase 2: Efficacy | Efficacy and fairness | Controlled implementation | - Small controlled environment - Build appropriate mechanisms for end-user interpretation, integration into clinical workflows - Include standards/thresholds for monitoring - Evaluate model prospectively and blind predictions to care team - Assess efficacy and fairness across subpopulation - Assess impact on quality and efficiency - Understand financial impact (anticipatory ahead of scaling) |
Phase 3: Effectiveness/Comparison to Existing Standard | Health outcomes | Larger study for effectiveness | - Larger deployment across multiple settings - Educate/training workforce - Determine comparative effectiveness between AI tool and standard of care - Ensure algorithm generalizability (geographical, domain, and temporal) - Ensure transparency with stakeholders about performance metrics |
Phase 4: Monitoring/Scaled and Post-Deployment Surveillance | Societal impact | Scaled and ongoing monitoring | - Machine learning operations (MLOps) - Monitor safety, workflow, equity, impact of AI systems - Broad dissemination - Publish findings and insights with communities to set benchmarks - Develop training programs to effectively integrate AI tools into clinical practice - Understand technical, cultural and patient care implications of ongoing technology evolution - Establish feedback loops to inform teams about clinical needs and ethical concerns |
