Fig. 3: AI-assisted scoring dynamics indicate improved fidelity of visual perception and higher scoring agreement potentially leading to increased positive diagnoses.
From: Augmented reality microscopy to bridge trust between AI and pathologists
a Comparison of manual vs AI-assisted scoring dynamics among 11 pathologists by cell type. Each square represents the change in scoring from manual (day 1) to AI-assisted (day 2) by each of the 11 pathologists and color gradations represent absolute cell counts where blue/red indicates less/more cells on day 2. Heatmaps: Top, changes in posTC; Middle, changes in negTC; Bottom, changes in posIC. b AI-assisted scoring increased awareness of absolute cell counts and improved fidelity of visual perception among pathologists. Tally of absolute cell counts by cell type: Left, negTC manual 361.9 (349-375) AI-assisted 905.7 (878-933); Middle, posTC manual 11.4 (10-13) AI-assisted 17.3 (15-19), Right, posIC manual 23.9 (22-26) AI-assisted 117.5 (109-127). c Agreement at PD-L1CPS ≥ 5 as evaluated by Fleiss’ kappa. Analysis of agreement among pathologists by Fleiss’ Kappa on day 1 using manual scoring Fleiss’ Kappa 0.52 (0.40-0.65) compared to day 2 using AI-assisted scoring Fleiss’ Kappa 0.76 (0.62-0.87) showing overall improved agreement with AI-assistance. d AI-assisted scoring prompted pathologists to identify 11 additional cases as positive for PD-L1 CPS ≥ 5. Consensus CPS ≥ 5 was calculated as the median score from the 11 individual pathologists. Manual scoring (day 1) identifies 15 cases of PD-L1 CPS > 5 vs 26 cases of PD-L1 CPS ≥ 5 by AI-assisted scoring (day 2) resulting in a 31% increase of positive cases.
