Table 3 Test precision, recall and specificity for the deep neural networks in COVID-19 detection (o.o.d. evaluation)a

From: Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization

Model and Metric

Normal

Pneumonia

COVID-19

Mean (macro-average)

ISNet precision

0.544 ± 0.026, [0.494,0.594]

0.794 ± 0.01, [0.774,0.814]

0.993 ± 0.002, [0.988,0.997]

0.777 ± 0.009, [0.759,0.795]

U-Net+DenseNet121 precision

0.446 ± 0.019, [0.408,0.483]

0.791 ± 0.015, [0.763,0.82]

0.723 ± 0.011, [0.702,0.744]

0.653 ± 0.009, [0.636,0.67]

DenseNet121 precision

0.364 ± 0.02, [0.324,0.402]

0.827 ± 0.018, [0.792,0.861]

0.649 ± 0.01, [0.629,0.67]

0.614 ± 0.009, [0.594,0.631]

Multi-task U-Net precision

0.552 ± 0.033, [0.488,0.617]

0.232 ± 0.02, [0.194,0.272]

0.469 ± 0.01, [0.449,0.489]

0.418 ± 0.013, [0.392,0.444]

AG-Sononet precision

0.104 ± 0.013, [0.079,0.129]

0.665 ± 0.025, [0.616,0.715]

0.549 ± 0.01, [0.528,0.569]

0.439 ± 0.01, [0.419,0.459]

Extended GAIN precision

0.189 ± 0.019, [0.152,0.225]

0.603 ± 0.016, [0.571,0.636]

0.642 ± 0.011, [0.62,0.664]

0.478 ± 0.009, [0.461,0.496]

RRR precision

0.262 ± 0.015, [0.232,0.293]

0.728 ± 0.016, [0.697,0.758]

0.723 ± 0.011, [0.701,0.745]

0.571 ± 0.008 [0.555,0.587]

Vision transformer (ViT-B/16) precision

0.268 ± 0.015, [0.239,0.297]

0.552 ± 0.016, [0.521,0.584]

0.572 ± 0.014, [0.544,0.598]

0.464 ± 0.009, [0.447,0.481]

ISNet recall

0.566 ± 0.026, [0.515,0.616]

0.933 ± 0.007, [0.919,0.946]

0.835 ± 0.01, [0.816,0.853]

0.778 ± 0.009, [0.76,0.796]

U-Net+DenseNet121 recall

0.796 ± 0.021, [0.756,0.837]

0.466 ± 0.014, [0.439,0.494]

0.838 ± 0.009, [0.819,0.856]

0.7 ± 0.009, [0.683,0.717]

DenseNet121 recall

0.57 ± 0.026, [0.518,0.618]

0.294 ± 0.013, [0.27,0.32]

0.916 ± 0.007, [0.902,0.93]

0.594 ± 0.01, [0.574,0.612]

Multi-task U-Net recall

0.338 ± 0.024, [0.29,0.386]

0.08 ± 0.008, [0.066,0.095]

0.776 ± 0.011, [0.755,0.797]

0.398 ± 0.009, [0.38,0.416]

AG-Sononet recall

0.156 ± 0.019, [0.12,0.192]

0.18 ± 0.011, [0.16,0.201]

0.824 ± 0.01, [0.805,0.843]

0.387 ± 0.008, [0.371,0.402]

Extended GAIN recall

0.22 ± 0.021, [0.178,0.261]

0.406 ± 0.014, [0.379,0.432]

0.796 ± 0.01, [0.775,0.816]

0.474 ± 0.009, [0.456,0.492]

RRR recall

0.574 ± 0.025, [0.524,0.624]

0.445 ± 0.014, [0.417,0.471]

0.753 ± 0.011, [0.731,0.775]

0.59 ± 0.01, [0.57,0.611]

Vision transformer (ViT-B/16) recall

0.665 ± 0.024, [0.616,0.712]

0.415 ± 0.014, [0.388,0.442]

0.486 ± 0.013, [0.461,0.511]

0.522 ± 0.01, [0.501,0.542]

ISNet specificity

0.937 ± 0.005, [0.928,0.946]

0.834 ± 0.009, [0.817,0.851]

0.995 ± 0.002, [0.991,0.998]

0.922 ± 0.003, [0.916,0.928]

U-Net+DenseNet121 specificity

0.869 ± 0.006, [0.857,0.882]

0.915 ± 0.006, [0.903,0.928]

0.708 ± 0.011, [0.686,0.73]

0.831 ± 0.004, [0.823,0.839]

DenseNet121 specificity

0.869 ± 0.006, [0.856,0.881]

0.958 ± 0.005, [0.949,0.967]

0.549 ± 0.012, [0.525,0.573]

0.792 ± 0.004, [0.784,0.8]

Multi-task U-Net specificity

0.964 ± 0.004, [0.957,0.971]

0.818 ± 0.009, [0.801,0.835]

0.201 ± 0.01, [0.182,0.22]

0.661 ± 0.004, [0.653,0.669]

AG-Sononet specificity

0.823 ± 0.007, [0.808,0.837]

0.938 ± 0.006, [0.927,0.948]

0.384 ± 0.012, [0.361,0.408]

0.715 ± 0.004, [0.707,0.722]

Extended GAIN specificity

0.875 ± 0.006, [0.863,0.888]

0.817 ± 0.009, [0.799,0.834]

0.597 ± 0.012, [0.573,0.62]

0.763 ± 0.004, [0.754,0.772]

RRR specificity

0.787 ± 0.008, [0.772,0.802]

0.886 ± 0.007, [0.872,0.9]

0.737 ± 0.011, [0.716,0.758]

0.803 ± 0.004, [0.795,0.812]

Vision transformer (ViT-B/16) specificity

0.761 ± 0.008, [0.745,0.776]

0.769 ± 0.01, [0.75,0.788]

0.669 ± 0.012, [0.646,0.692]

0.733 ± 0.005, [0.723,0.742]

  1. aMetrics are reported as: mean ± std, [95% HDI], according to Bayesian estimation. Supplementary Note 10 provides more details about the statistical analysis.