Table 2 Performance of models for predicting preterm birth at 10% FPR using tenfold cross-validation, by classification algorithms.

From: Development of prognostic model for preterm birth using machine learning in a population-based cohort of Western Australia births between 1980 and 2015

Model

Algorithm

Evaluation metrics

AUC, %

Accuracy, %

F1, %

TPR, %

PPV, %

NPV, %

LR + 

LR-

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

Mean (95% CI)

A

LR

83.56 (83.35,83.77)

87.43 (87.41,87.46)

44.97 (44.81,45.14)

60.01 (59.72,60.29)

35.96 (35.85,36.07)

96.01 (95.98,96.03)

6.00 (5.97,6.03)

0.44 (0.44,0.45)

DT

82.79 (82.62,82.96)

87.49 (87.44,87.55)

45.63 (45.48,45.78)

61.32 (61.13,61.51)

36.33 (36.19,36.48)

96.13 (96.11,96.15)

6.10 (6.06,6.14)

0.43 (0.43,0.43)

RF

84.24 (84.05,84.43)

87.57 (87.54,87.60)

45.90 (45.72,46.08)

61.62 (61.30,61.93)

36.57 (36.45,36.69)

96.16 (96.13,96.19)

6.16 (6.13,6.19)

0.43 (0.42,0.43)

XGB

84.34 (84.15,84.53)

87.62 (87.59,87.65)

46.23 (46.05,46.41)

62.19 (61.89,62.50)

36.79 (36.67,36.91)

96.22 (96.19,96.25)

6.22 (6.19,6.25)

0.42 (0.42,0.42)

MLP

84.40 (84.21,84.59)

87.64 (87.61,87.67)

46.35 (46.14,46.57)

62.41 (62.04,62.78)

36.87 (36.73,37.01)

96.24 (96.20,96.27)

6.24 (6.20,6.28)

0.42 (0.41,0.42)

B

LR

85.88 (85.71,86.05)

87.90 (87.85,87.94)

46.44 (46.13,46.74)

64.25 (63.70,64.80)

36.36 (36.16,36.55)

96.59 (96.54,96.64)

6.42 (6.37,6.48)

0.40 (0.39,0.40)

DT

83.42 (83.22,83.62)

87.87 (87.79,87.96)

46.28 (45.98,46.57)

63.96 (63.39,64.54)

36.25 (36.02,36.49)

96.56 (96.51,96.61)

6.40 (6.33,6.46)

0.40 (0.39,0.41)

RF

85.44 (85.27,85.61)

87.95 (87.90,88.00)

46.79 (46.46,47.12)

64.89 (64.29,65.48)

36.59 (36.37,36.80)

96.65 (96.59,96.70)

6.49 (6.43,6.55)

0.39 (0.38,0.40)

XGB

86.22 (86.04,86.40)

88.00 (87.95,88.05)

47.14 (46.80,47.49)

65.53 (64.89,66.16)

36.81 (36.59,37.04)

96.71 (96.65,96.76)

6.55 (6.49,6.61)

0.38 (0.38,0.39)

MLP

86.41 (86.20,86.62)

88.06 (88.01,88.11)

47.53 (47.22,47.84)

66.24 (65.68,66.79)

37.06 (36.86,37.27)

96.77 (96.72,96.82)

6.62 (6.56,6.68)

0.38 (0.37,0.38)

C

LR

83.67 (83.03,84.31)

87.14 (87.02,87.26)

46.86 (46.12,47.60)

59.79 (58.57,61.01)

38.53 (38.04,39.02)

95.53 (95.40,95.66)

5.98 (5.86,6.11)

0.45 (0.43,0.46)

DT

81.36 (80.64,82.08)

87.61 (87.35,87.87)

46.44 (45.55,47.34)

56.61 (55.63,57.60)

39.38 (38.51,40.24)

95.23 (95.13,95.34)

6.21 (5.98,6.43)

0.48 (0.47,0.49)

RF

83.13 (82.57,83.69)

87.16 (87.03,87.28)

46.97 (46.20,47.75)

60.00 (58.71,61.29)

38.60 (38.09,39.11)

95.55 (95.41,95.69)

6.00 (5.87,6.13)

0.44 (0.43,0.46)

XGB

84.11 (83.49,84.73)

87.29 (87.17,87.41)

47.76 (47.00,48.52)

61.28 (59.99,62.57)

39.13 (38.63,39.63)

95.69 (95.55,95.82)

6.14 (6.01,6.27)

0.43 (0.42,0.44)

MLP

83.01 (82.41,83.61)

87.11 (86.99,87.23)

46.67 (45.90,47.44)

59.49 (58.21,60.76)

38.40 (37.89,38.91)

95.50 (95.36,95.63)

5.95 (5.82,6.08)

0.45 (0.44,0.46)

D

LR

58.20 (58.04,58.36)

83.83 (83.80,83.85)

15.98 (15.78,16.18)

17.95 (17.71,18.20)

14.40 (14.23,14.57)

92.13 (92.11,92.15)

1.80 (1.77,1.82)

0.91 (0.91,0.91)

DT

57.76 (57.59,57.93)

83.90 (83.85,83.96)

16.16 (16.05,16.28)

18.12 (17.93,18.30)

14.59 (14.52,14.67)

92.15 (92.14,92.16)

1.82 (1.81,1.83)

0.91 (0.91,0.91)

RF

58.43 (58.27,58.59)

83.87 (83.85,83.89)

16.37 (16.18,16.56)

18.42 (18.19,18.66)

14.72 (14.57,14.88)

92.17 (92.15,92.19)

1.84 (1.82,1.87)

0.91 (0.90,0.91)

XGB

58.70 (58.52,58.88)

83.89 (83.86,83.91)

16.58 (16.37,16.79)

18.69 (18.43,18.95)

14.89 (14.72,15.07)

92.20 (92.17,92.22)

1.87 (1.84,1.89)

0.90 (0.90,0.91)

MLP

58.69 (58.53,58.85)

83.88 (83.86,83.91)

16.59 (16.41,16.78)

18.71 (18.49,18.94)

14.90 (14.75,15.06)

92.20 (92.18,92.22)

1.87 (1.85,1.89)

0.90 (0.90,0.91)

E

LR

69.01 (68.66,69.36)

85.47 (85.43,85.51)

28.04 (27.70,28.39)

34.63 (34.14,35.12)

23.56 (23.31,23.82)

93.92 (93.88,93.97)

3.46 (3.41,3.51)

0.73 (0.72,0.73)

DT

66.96 (66.64,67.28)

85.53 (85.46,85.61)

27.23 (26.84,27.62)

33.11 (32.43,33.80)

23.13 (22.87,23.38)

93.81 (93.75,93.86)

3.38 (3.33,3.43)

0.74 (0.73,0.75)

RF

67.71 (67.45,67.97)

85.39 (85.36,85.43)

27.36 (27.06,27.65)

33.64 (33.21,34.06)

23.05 (22.83,23.27)

93.84 (93.80,93.88)

3.36 (3.32,3.41)

0.74 (0.73,0.74)

XGB

69.24 (68.91,69.57)

85.50 (85.46,85.55)

28.35 (27.98,28.71)

35.07 (34.54,35.60)

23.79 (23.52,24.06)

93.96 (93.92,94.01)

3.51 (3.45,3.56)

0.72 (0.72,0.73)

MLP

69.14 (68.80,69.48)

85.50 (85.45,85.54)

28.24 (27.86,28.61)

34.89 (34.35,35.43)

23.71 (23.44,23.99)

93.95 (93.90,94.00)

3.49 (3.44,3.54)

0.72 (0.72,0.73)

F

LR

85.53 (85.22,85.84)

87.85 (87.79,87.91)

46.14 (45.75,46.54)

63.69 (62.97,64.41)

36.18 (35.92,36.43)

96.53 (96.47,96.60)

6.37 (6.30,6.44)

0.40 (0.40,0.41)

DT

83.75 (83.42,84.08)

87.75 (87.65,87.85)

45.84 (45.46,46.23)

63.43 (62.73,64.13)

35.89 (35.59,36.19)

96.51 (96.44,96.57)

6.29 (6.21,6.38)

0.41 (0.40,0.41)

RF

85.35 (85.04,85.66)

87.91 (87.85,87.97)

46.54 (46.14,46.94)

64.42 (63.69,65.16)

36.43 (36.17,36.69)

96.60 (96.53,96.67)

6.44 (6.37,6.51)

0.40 (0.39,0.40)

XGB

85.63 (85.32,85.94)

87.89 (87.83,87.95)

46.41 (46.03,46.80)

64.19 (63.50,64.87)

36.35 (36.10,36.60)

96.58 (96.52,96.64)

6.42 (6.35,6.49)

0.40 (0.39,0.41)

MLP

85.68 (85.32,86.04)

87.91 (87.87,87.96)

46.57 (46.23,46.91)

64.45 (63.83,65.07)

36.46 (36.24,36.67)

96.60 (96.55,96.66)

6.45 (6.39,6.51)

0.40 (0.39,0.40)

  1. AUC, area under the receiving-operator characteristic curve; TPR, true positive rate (recall or sensitivity); PPV, positive predictive value (precision); NPV, negative predictive value; LR + , positive likelihood ratio; LR − , negative likelihood ratio; CI, confidence interval; FPR, false positive rate (1-specificity); LR, regularised logistic regression; DT, decision trees; RF, Random Forests; XGB, extreme gradient boosting; MLP, multi-layer perceptron.
  2. Model A: Cohort—all births; Predictors—maternal socio-demographic factors, maternal chronic medical conditions, and current pregnancy characteristics and complications; Model B: Cohort—births of multiparous women; Predictors—Model A + maternal past obstetric history; Model C: Cohort—births of parents who were born during the study period; Predictors—Model A + parent’s birth outcomes and grandmother’s chronic medical conditions and obstetric history; Model D: Cohort—all births; Predictors—maternal socio-demographic factors, maternal chronic medical conditions, parity, and birth year; Model E: Cohort—births of multiparous women; Predictors—Model D + maternal past obstetric history; Model F: Cohort—births of multiparous women; Predictors—Model B excluding small-for-gestational age and congenital anomalies in current birth.
Back to article page