Table 3 Cancer detection on rare prostate morphologies (Test set 6)

From: Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction

 

Benign mimics of prostate cancer

Rare prostate cancer subtypes

Confidence 99.9%

Adenosis, n = 36

Basal cell hyperplasia, n = 21

Clearcell cribriform hyperplasia, n = 3

Prostatic atrophy, n = 37

Post-atrophic hyperplasia, n = 5

Cowper’s glands, n = 6

Atrophy like cancer, n = 11

Foamy, n = 13

PIN like cancer, n = 3

Pseudo-hyper-plastic, n = 41

Small-cell cancer, n = 3

All cases, n = 179

Conformal prediction regions for cancer detection

            

Error, n (%)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

2 (33%)

1 (9%)

0 (0)

0 (0)

0 (0)

0 (0)

3 (2%)

Empty, n (%)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

Single predictions, n (%)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

0 (0)

4 (36%)

9 (69%)

0 (0)

19 (46%)

1 (33%)

33 (18%)

Multiple predictions, n (%)

36 (100%)

21 (100%)

3 (100%)

37 (100%)

5 (100%)

4 (67%)

6 (55%)

4 (31%)

3 (100%)

22 (54%)

2 (67%)

143 (80%)

AI point predictions for cancer detection

            

Error, n (%)

16 (44%)

1 (5%)

0 (0)

9 (24%)

3 (60%)

5 (83%)

3 (27%)

0 (0)

2 (67%)

4 (10%)

1 (33%)

44 (25%)

Correct, n (%)

20 (56%)

20 (95%)

3 (100%)

28 (76%)

2 (40%)

1 (17%)

8 (73%)

13 (100%)

1 (33%)

37 (90%)

2 (67%)

135 (75%)

  1. The results are presented both as prediction regions by the conformal predictor and point predictions by the AI system without the conformal predictor. The conformal prediction regions are reported at a confidence level of 99.9%. The error is the fraction of true labels not included in the prediction. A multi-label prediction means that the prediction is uncertain, and the model cannot distinguish between several possible class labels at the desired confidence. Empty set predictions are examples where the model could not assign any label, typically meaning that the example was very different from the data the model was trained on. These tissue sections contain morphologies that are typically difficult to grade for pathologists, Including benign mimics of prostate cancer; adenosis, n = 36, basal cell hyperplasia, n = 21, clear cell cribriform hyperplasia, n = 3, prostatic atrophy, n = 37, postatrophic hyperplasia, n = 5, Cowperʼs glands, n = 6 and rare prostate cancer subtypes; atrophy like cancer, n = 11, foamy gland, n = 13, PIN-like cancer, n = 3, pseudohyperplastic cancer, 41, small-cell cancer, n = 3.