Table 1 Overview of the performance of our model, the pathologists, and previous models.

From: Deep neural network trained on gigapixel images improves lymph node metastasis detection in clinical settings

Model / Pathologist

TP

FP

FN

TN

Sensitivity

Specificity

MCC

Our model

263

12

32

849

0.8915(0.8503–0.9246)

0.9861(0.9758–0.9928)

0.8986(0.8686–0.9269)

Pathologist S.-C.H.

289

2

6

859

0.9797(0.9563–0.9925)

0.9977(0.9916–0.9997)

0.9818(0.9681–0.9932)

Pathologist 1

279

2

16

859

0.9458(0.9134–0.9687)

0.9977(0.9916–0.9997)

0.9589(0.9392–0.9767)

Pathologist 2

286

11

9

850

0.9695(0.9429–0.9860)

0.9872(0.9773–0.9936)

0.9546(0.9340–0.9731)

Pathologist 3

275

2

20

859

0.9322(0.8972–0.9581)

0.9977(0.9916–0.9997)

0.9497(0.9284–0.9690)

Pathologist 1with partial AI assistance

289

2

6

859

0.9797(0.9563–0.9925)

0.9977(0.9916–0.9997)

0.9818(0.9682–0.9932)

Pathologist 2with partial AI assistance

291

2

4

859

0.9864(0.9656–0.9963)

0.9977(0.9916–0.9997)

0.9863(0.9742–0.9956)

Pathologist 3with partial AI assistance

288

2

7

859

0.9763(0.9517–0.9904)

0.9977(0.9916–0.9997)

0.9795(0.9648–0.9912)

Hu et al.

159

11

21

1025

0.8833(0.8272–0.9263)

0.9894(0.9811–0.9947)

0.8937(0.8566–0.9283)

Wang et al.

5217

391

82

9544

0.9845 (0.9808–0.9877)

0.9606 (0.9566–0.9644)

0.9334(0.9275–0.9391)

  1. The confusion matrices were calculated for our model (at a threshold of 0.4) and the pathologists, including the number of true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) LN images under the main test set (n = 1156). Three pathologists (J.L., H.-C.C., and T.-Y.H.) relabeled the 38 equivocal LN images with AI assistance (denoted as partial AI assistance). The data on model performance reported in the bottom two rows of the table were directly retrieved from the publications in question. Considering the between-study discrepancies in test slide distributions, the results may contain bias. MCC is an abbreviation for Matthews correlation coefficient. Supplementary Table 1 provides extended information, including additional metrics and model performance results on the micrometastasis and ITC test subsets.
Back to article page