Fig. 4: Experimental results.
From: Biological data annotation via a human-augmenting AI-based labeling system
a The average workload reduction and effectiveness increase for the four use-cases considered (TILs, tumor cells, eosinophils, and Ki-67 cells), altogether averaging a workload reduction of 90.6% and a 4.34% effectiveness improvement. Workload fraction is measured by the fraction of AI predictions changed by the annotator (0 is perfect model annotation, 1 is without an AI). Workload reduction is the inverse (1–workload fraction). Effectiveness is measured as the area under the accuracy vs number of samples (N) plot, bounded by N < 200, for an AI model trained on the resultant annotated data set (see example in Fig. 5). b Workload results on the seven tested annotators–top histopathology trainees from Stanford and the University of California, San Francisco. c Effectiveness results on the same set of trainees.
