Table 3 Comparison of the diagnostic performance between the ECMAI-ME model and endoscopists for diagnosing Gastrointestinal stromal tumors.
Accuracy (%, CI) | P value† | Sensitivity (%, CI) | P value† | Specificity (%, CI) | P value† | PPV (%, CI) | NPV (%, CI) | |
|---|---|---|---|---|---|---|---|---|
AI model vs. Endoscopists (N = 163) | ||||||||
ECMAI-ME | 85.28 (79.84–90.72) | / | 76.92 (65.47–88.37) | / | 89.19 (83.41–94.97) | / | 76.92 (65.47–88.37) | 89.19 (83.41–94.97) |
Expert 1 | 77.91 (71.55–84.28) | <0.001*** | 90.38 (82.37–98.40) | 0.092 | 72.07 (63.73–80.42) | <0.001*** | 60.26 (49.40-71.12) | 94.12 (89.12–99.12) |
Expert 2 | 76.79 (70.20-83.18) | <0.001*** | 84.62 (74.81–94.42) | 0.388 | 72.97 (64.71–81.23) | <0.001*** | 59.46 (48.27–70.65) | 91.01 (85.07–96.95) |
Novice 1 | 64.42 (57.07–71.77) | <0.001*** | 82.69 (72.41–92.97) | 0.629 | 55.86 (46.62–65.09) | <0.001*** | 46.74 (36.54–56.93) | 87.32 (79.58–95.06) |
Novice 2 | 70.55 (63.55–77.55) | 0.038* | 67.31 (54.56–80.06) | 0.267 | 72.07 (63.73–80.42) | <0.001*** | 55.56 (43.29–67.83) | 83.17 (75.87–90.47) |
Trainee 1 | 61.35 (53.87–68.83) | <0.001*** | 76.92 (65.47–88.37) | 1.000 | 54.05 (44.78–63.33) | <0.001*** | 43.96 (33.76–54.15) | 83.33 (74.72–91.94) |
Trainee 2 | 56.44 (48.83–64.05) | <0.001*** | 65.38 (52.45–78.32) | 0.263 | 52.25 (42.96–61.54) | <0.001*** | 39.08 (28.83–49.33) | 76.32 (66.76–85.87) |
AI model vs. initial diagnosis in the original reports (N = 152) ‡ | ||||||||
ECMAI-ME | 84.87 (79.17–90.57) | / | 78.00 (66.52–89.48) | / | 88.24 (81.98–94.49) | / | 76.47 (64.83–88.11) | 89.11 (83.03–95.18) |
Original reports | 78.95 (72.47–85.43) | <0.001*** | 92.00 (84.48–99.52) | 0.118 | 72.55 (63.89–81.21) | 0.002** | 62.16 (51.11–73.21) | 94.87 (89.98–99.77) |
