Table 4 Areas under the receiver operating characteristic curve (AUROCs) for NLP models that interpret medical oncologist progress note text to ascertain clinical outcomes, as evaluated in the labeled test set.
From: Artificial intelligence-aided clinical annotation of a large multi-cancer genomic dataset
Clinical outcome | Models trained on all cancer types | |||||||
|---|---|---|---|---|---|---|---|---|
All patients | Breast cancer | Colorectal cancer | NSCLC | Pancreatic cancer | Prostate cancer | Renal cell carcinoma | Urothelial carcinoma | |
Any cancer | 0.93 | 0.98 | 0.98 | 0.95 | 0.78 | 0.91 | 0.91 | 0.95 |
Progression | 0.92 | 0.97 | 0.91 | 0.96 | 0.92 | 0.87 | 0.86 | 0.78 |
Response | 0.93 | 0.96 | 0.95 | 0.95 | 0.91 | 0.87 | 0.89 | 0.99 |
Models trained on all cancers except for the type under evaluation | ||||||||
Any cancer | † | 0.98 | 0.97 | 0.95 | 0.79 | 0.85 | 0.90 | 0.95 |
Progression | † | 0.96 | 0.90 | 0.95 | 0.91 | 0.84 | 0.84 | 0.78 |
Response | † | 0.94 | 0.94 | 0.94 | 0.88 | 0.83 | 0.88 | 1.0 |