Table 2 Performance for detecting Clostridioides difficile infection

From: Deep learning-based prediction of Clostridioides difficile infection caused by antibiotics using longitudinal electronic health records

 

AUROC

P-value

AUPRC

Sensitivity

Specificity

Precision

F1-score

Internal validation

       

Tree-based model

       

Random forest

0.834

(0.791–0.877)

 

0.049

(0.006–0.092)

0.660

(0.651–0.668)

0.875

(0.870–0.881)

0.037

(0.034–0.040)

0.070

(0.065–0.074)

GBM

0.853

(0.813–0.894)

0.1493

0.070

(0.029–0.111)

0.702

(0.694–0.710)

0.868

(0.862–0.874)

0.037

(0.034–0.040)

0.070

(0.066–0.075)

RNN-based model

       

Simple RNN

0.968

(0.957–0.979)

<0.0001

0.165

(0.154–0.176)

0.936

(0.932–0.940)

0.877

(0.871–0.883)

0.052

(0.048–0.056)

0.099

(0.094–0.104)

LSTM

0.939

(0.918–0.961)

0.0001

0.118

(0.096–0.140)

0.894

(0.888–0.899)

0.891

(0.885–0.896)

0.056

(0.052–0.060)

0.105

(0.099–0.110)

GRU

0.952

(0.932–0.973)

<0.0001

0.250

(0.229–0.270)

0.936

(0.932–0.940)

0.862

(0.856–0.867)

0.046

(0.043–0.050)

0.088

(0.084–0.093)

Attention-based model

       

Transformer

0.871

(0.837–0.904)

0.1053

0.074

(0.040–0.108)

0.755

(0.748–0.763)

0.839

(0.833–0.845)

0.033

(0.030–0.036)

0.063

(0.059–0.067)

RETAIN

0.746

(0.699–0.793)

0.0271

0.025

(0.000–0.072)

0.777

(0.769–0.784)

0.634

(0.626–0.642)

0.015

(0.013–0.017)

0.030

(0.027–0.033)

External validation

       

Tree-based model

       

Random forest

0.833

(0.818–0.847)

 

0.086

(0.071–0.100)

0.808

(0.804–0.813)

0.742

(0.738–0.747)

0.060

(0.057–0.063)

0.112

(0.108–0.115)

GBM

0.860

(0.847–0.873)

<0.0001

0.118

(0.105–0.131)

0.773

(0.768–0.777)

0.799

(0.795–0.804)

0.073

(0.070–0.075)

0.133

(0.129–0.136)

RNN-based model

       

Simple RNN

0.958

(0.953–0.963)

<0.0001

0.241

(0.237–0.246)

0.935

(0.932–0.937)

0.881

(0.878–0.885)

0.138

(0.134–0.142)

0.241

(0.236–0.245)

LSTM

0.940

(0.934–0.947)

<0.0001

0.271

(0.264–0.277)

0.914

(0.911–0.917)

0.831

(0.827–0.836)

0.099

(0.096–0.103)

0.179

(0.175–0.183)

GRU

0.972

(0.968–0.975)

<0.0001

0.535

(0.531–0.539)

0.938

(0.935–0.940)

0.884

(0.881–0.888)

0.141

(0.138–0.145)

0.246

(0.241–0.251)

Attention-based model

       

Transformer

0.871

(0.858–0.883)

<0.0001

0.179

(0.166–0.191)

0.807

(0.803–0.811)

0.786

(0.781–0.790)

0.071

(0.068–0.074)

0.131

(0.127–0.134)

RETAIN

0.755

(0.735–0.774)

<0.0001

0.091

(0.071–0.110)

0.572

(0.566–0.577)

0.803

(0.799–0.808)

0.056

(0.053–0.058)

0.102

(0.098–0.105)

  1. Bold indicates the best and underline indicates the second best.
  2. P-values were calculated using the DeLong method. Sensitivity, specificity, precision, and F1-score were calculated using Youden’s index.
  3. AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve, GBM gradient boosting machine, RNN recurrent neural network, LSTM long short-term memory, GRU gated recurrent unit.
Back to article page