Table 7 Clinical trial device performance vs. real world performance.
From: An analysis of the real world performance of an artificial intelligence based autism diagnostic
Metric | Demographic | Clinical trial performance | Real world performance | Fisher’s Exact Test p value |
|---|---|---|---|---|
PPV | All Demographics | 80.8% (n = 78) | 92.4% (n = 119) | 0.024* (n = 197) |
Female | 60.0% (n = 20) | 85.7% (n = 35) | 0.048* (n = 55) | |
Male | 87.9% (n = 58) | 95.2% (n = 85) | 0.12 (n = 142) | |
Under 48 months of age | 81.5% (n = 65) | 96.3% (n = 82) | 0.005* (n = 147) | |
Over 48 months of age | 76.9% (n = 13) | 83.8% (n = 37) | 0.68 (n = 50) | |
NPV | All Demographics | 98.2% (n = 57) | 97.6% (n = 41) | 1.0 (n = 98) |
Female | 96.0% (n = 25) | 100.0% (n = 13) | 1.0 (n = 38) | |
Male | 100.0% (n = 32) | 96.4% (n = 28) | 0.47 (n = 60) | |
Under 48 months of age | 97.8% (n = 45) | 96.6% (n = 29) | 1.0 (n = 74) | |
Over 48 months of age | 100.0% (n = 12) | 100.0% (n = 12) | 1.0 (n = 24) | |
Determinate rate | All Demographics | 31.8% (n = 425) | 63.0% (n = 254) | < 0.001* (n = 679) |
Female | 29.2% (n = 154) | 64.9% (n = 74) | < 0.001* (n = 228) | |
Male | 33.2% (n = 451) | 62.2% (n = 180) | < 0.001* (n = 451) | |
Under 48 months of age | 38.1% (n = 289) | 66.1% (n = 168) | < 0.001* (n = 457) | |
Over 48 months of age | 18.4% (n = 136) | 57.0% (n = 86) | < 0.001* (n = 222) |
