Table 1 Quality and safety performance by ERNIE (N = 384)

From: Quality safety and disparity of an AI chatbot in managing chronic diseases: simulated patient experiments

 

Unstable angina

(n = 192)

Asthma

(n = 192)

Total

(n = 384)

 

Mean

95% CI

Mean

95% CI

Mean

95% CI

Quality indicators

 % completion of the full checklists (0-1)

0.176

0.166, 0.186

0.115

0.106, 0.123

0.145

0.138, 0.153

 % completion of the essential checklists (core, subset, 0-1)

0.354

0.336, 0.372

0.051

0.041, 0.061

0.203

0.184, 0.221

 % Correct diagnosis (0-1)

0.766

0.705, 0.826

0.781

0.722, 0.840

0.773

0.731, 0.815

 % Correct medication (0-1)

0.948

0.916, 0.980

0.938

0.903, 0.972

0.943

0.919, 0.966

Safety

 No. of tests requested

3.09

2.91, 3.27

3.10

2.90, 3.30

3.09

2.96, 3.23

 % Unnecessary test requested (0-1)

0.969

0.944, 0.994

0.870

0.822, 0.918

0.919

0.892, 0.947

 No. of medication prescribed

3.97

3.69, 4.26

4.21

3.91, 4.51

4.09

3.89, 4.30

 % Inappropriate medication prescribed (0-1)

0.526

0.455, 0.597

0.630

0.561, 0.699

0.578

0.529, 0.628

  1. Note. Means and 95% confidence intervals (CIs) for binary and continuous variables.