Fig. 2: Comparison of source data model receiver operator curves (ROC), estimated external validation ROC, and observed external validation ROC on 13 datasets and 5 modalities. | npj Digital Medicine

Fig. 2: Comparison of source data model receiver operator curves (ROC), estimated external validation ROC, and observed external validation ROC on 13 datasets and 5 modalities.

From: Shortcut learning in medical AI hinders generalization: method for estimating AI model generalization without external data

Fig. 2

Model receiver operator characteristic curves on source (a, c, e, g, i, k, m, o, q) and external validation datasets (b, d, f, h, j, l, n, p, r, s, t). Source dataset figures include the corresponding DABIS estimate (gray) and the external dataset figures include our estimated curves (yellow). Shaded regions depict the 95% confidence interval. Notice that the ROC curves on the external test datasets (green) are much better approximated by our predicted curves (yellow) than they are by the traditional source test dataset curves (red). MIMIC-CXR was shortened to CXR in this figure.

Back to article page