Table 3 Threshold specific performance metrics for predicting exacerbation within 2 days (validation cohort).

From: Machine learning did not beat logistic regression in time series prediction for severe asthma exacerbations

Probability threshold

Model

Sensitivity

Specificity

PPV

NPV

0.001

XGBoost

0.71 (133/188)

0.81 (32,178/39,904)

0.02 (133/7859)

1.0 (32,178/32,233)

Logistic regression

0.93 (174/188)

0.56 (22,227/39,904)

0.01 (174/17,851)

1.0 (22,227/22,241)

0.002

XGBoost

0.59 (110/188)

0.89 (35,326/39,904)

0.02 (110/4688)

1.0 (35,326/35,404)

Logistic regression

0.84 (158/188)

0.82 (32,720/39,904)

0.02 (158/7342)

1.0 (32,720/32,750)

Resulting in 5217 positive predictionsb

One class SVM

0.34 (64/188)

0.87 (34,751/39,904)

0.01 (64/5217)

1.0 (34,751/34,875)

XGBoost

0.6 (112/188)

0.87 (34,800/39,904)

0.02 (112/5216)

1.0 (34,800/34,876)

Logistic regression

0.73 (137/188)

0.87 (34,823/39,904)

0.03 (137/5218)

1.0 (34,823/34,874)

Resulting in 138 positive predictionsb

Clinical rulea

0.05 (10/188)

1.0 (39,776/39,904)

0.07 (10/138)

1.0 (39,776/39,954)

XGBoost

0.11 (21/188)

1.0 (39,787/39,904)

0.15 (21/138)

1.0 (39,787/39,954)

Logistic regression

0.11 (20/188)

1.0 (39,787/39,904)

0.15 (20/137)

1.0 (39,787/39,955)

  1. SVM support vector machine, XGBoost gradient boosted decision trees, ppv positive predictive value, NPV negative predictive value.
  2. aPeak Expiratory Flow < 60% personal best.
  3. bThis threshold is set so that the XGBoost and logistic regression models produce the same number of positive predictions as the one class SVM or clinical rule.
Back to article page