Table 2 Performance of AI models in assessing tumor-infiltrating NTR
Model | AUC (95% CI) | ACC (95% CI) | Sensitivity | Specificity | F1-score |
|---|---|---|---|---|---|
Cuboid-model | |||||
 Training | 0.88 (0.83–0.93) | 0.82 (0.76−0.88) | 0.90 | 0.74 | 0.83 |
 Internal validation | 0.85 (0.77−0.93) | 0.82 (0.74−0.90) | 0.84 | 0.80 | 0.82 |
 External validation | 0.81 (0.71−0.91) | 0.77 (0.67−0.87) | 0.82 | 0.71 | 0.80 |
ROIex-3mm-model | |||||
 Training | 0.89 (0.84−0.94) | 0.83 (0.77−0.89) | 0.90 | 0.76 | 0.84 |
 Internal validation | 0.84 (0.76−0.92) | 0.84 (0.77−0.91) | 0.88 | 0.80 | 0.85 |
 External validation | 0.83 (0.70−0.96) | 0.77 (0.64−0.90) | 0.82 | 0.71 | 0.80 |
ROIex-5mm-model | |||||
 Training | 0.84 (0.78−0.90) | 0.77 (0.71−0.83) | 0.84 | 0.70 | 0.79 |
 Internal validation | 0.78 (0.68−0.88) | 0.76 (0.67−0.85) | 0.76 | 0.76 | 0.76 |
 External validation | 0.76 (0.65−0.87) | 0.72 (0.61−0.82) | 0.77 | 0.65 | 0.76 |
ROI-model | |||||
 Training | 0.90 (0.85−0.95) | 0.86 (0.81−0.91) | 0.92 | 0.80 | 0.87 |
 Internal validation | 0.88 (0.81−0.95) | 0.84 (0.76−0.92) | 0.84 | 0.84 | 0.84 |
 External validation | 0.85 (0.76−0.94) | 0.79 (0.70−0.89) | 0.82 | 0.76 | 0.82 |
Multimodal Cuboid-model | |||||
 Training | 0.91 (0.86−0.95) | 0.87 (0.82−0.92) | 0.92 | 0.82 | 0.88 |
 Internal validation | 0.89 (0.82−0.96) | 0.86 (0.79−0.93) | 0.88 | 0.84 | 0.86 |
 External validation | 0.86 (0.77−0.95) | 0.82 (0.73−0.91) | 0.86 | 0.76 | 0.84 |
Multimodal ROIex-model | |||||
 Training | 0.89 (0.84−0.94) | 0.87 (0.82−0.92) | 0.90 | 0.84 | 0.87 |
 Internal validation | 0.87 (0.79−0.95) | 0.84 (0.76−0.92) | 0.88 | 0.80 | 0.85 |
 External validation | 0.84 (0.75−0.93) | 0.79 (0.70−0.89) | 0.86 | 0.71 | 0.83 |
Multimodal ROI-model (PORCELAIN) | |||||
 Training | 0.94 (0.89–0.97) | 0.89 (0.84−0.94) | 0.94 | 0.84 | 0.90 |
 Internal validation | 0.92 (0.86−0.98) | 0.88 (0.81−0.95) | 0.88 | 0.88 | 0.88 |
 External validation | 0.90 (0.84−0.94) | 0.85 (0.76−0.93) | 0.86 | 0.82 | 0.86 |