Table 3 Comparison of various diagnostic accuracy metrics (with 95% CI) for the prognostic models under consideration.
From: Prognostication and Risk Factors for Cystic Fibrosis via Automated Machine Learning
Prognostic model | AUC-ROC | Youden’s J statistic | AUC-PR | Average Precision | F1 score |
|---|---|---|---|---|---|
AutoPrognosis | 0.89 ± 0.01 | 0.67 ± 0.02 | 0.58 ± 0.04 | 0.59 ± 0.04 | 0.60 ± 0.03 |
Nkam et al.36 | 0.86 ± 0.01 | 0.58 ± 0.03 | 0.50 ± 0.03 | 0.48 ± 0.03 | 0.52 ± 0.02 |
Buzzetti et al.23 | 0.83 ± 0.01 | 0.54 ± 0.03 | 0.42 ± 0.02 | 0.44 ± 0.03 | 0.49 ± 0.02 |
CF-ABLE-UK40 | 0.77 ± 0.01 | 0.48 ± 0.05 | 0.28 ± 0.04 | 0.20 ± 0.02 | 0.34 ± 0.02 |
FEV1% predicted criterion15 | 0.70 ± 0.01 | 0.41 ± 0.02 | 0.50 ± 0.02 | 0.27 ± 0.02 | 0.47 ± 0.01 |
SVM | 0.84 ± 0.03 | 0.60 ± 0.05 | 0.50 ± 0.09 | 0.51 ± 0.09 | 0.52 ± 0.07 |
Gradient Boosting | 0.87 ± 0.02 | 0.63 ± 0.01 | 0.55 ± 0.03 | 0.55 ± 0.04 | 0.56 ± 0.01 |
Bagging | 0.83 ± 0.03 | 0.58 ± 0.05 | 0.51 ± 0.04 | 0.47 ± 0.04 | 0.52 ± 0.03 |
Pipeline 1 (grid search) | 0.83 ± 0.02 | 0.56 ± 0.03 | 0.51 ± 0.04 | 0.47 ± 0.04 | 0.51 ± 0.03 |
Pipeline 1 (random search) | 0.84 ± 0.01 | 0.56 ± 0.02 | 0.53 ± 0.02 | 0.49 ± 0.032 | 0.53 ± 0.02 |
Pipeline 2 (grid search) | 0.87 ± 0.03 | 0.62 ± 0.02 | 0.54 ± 0.05 | 0.55 ± 0.03 | 0.57 ± 0.01 |
Pipeline 2 (random search) | 0.83 ± 0.02 | 0.56 ± 0.03 | 0.51 ± 0.04 | 0.47 ± 0.04 | 0.51 ± 0.03 |
TPOT | 0.84 ± 0.01 | 0.56 ± 0.03 | 0.51 ± 0.02 | 0.49 ± 0.02 | 0.51 ± 0.02 |
