Table 3 Areas under the receiver operating characteristic curve (AUROCs) for NLP models that interpret imaging report text to ascertain clinical outcomes, as evaluated in the labeled test set.
From: Artificial intelligence-aided clinical annotation of a large multi-cancer genomic dataset
Clinical outcome | Models trained on imaging reports for patients with all listed cancer types | |||||||
|---|---|---|---|---|---|---|---|---|
All patients | Breast cancer | Colorectal cancer | NSCLC | Pancreatic cancer | Prostate cancer | Renal cell carcinoma | Urothelial carcinoma | |
Any cancer | 0.98 | 0.98 | 0.98 | 0.97 | 0.97 | 0.98 | 0.97 | 0.97 |
Progression | 0.95 | 0.95 | 0.96 | 0.96 | 0.93 | 0.96 | 0.96 | 0.94 |
Response | 0.97 | 0.98 | 0.99 | 0.96 | 0.97 | 0.94 | 0.97 | 0.99 |
Brain metastasis | 0.99 | 0.97 | 1.0 | 0.99 | 1.0 | 0.98 | 0.97 | * |
Bone metastasis | 0.99 | 0.98 | 0.99 | 0.98 | 0.99 | 0.98 | 0.99 | 0.99 |
Adrenal metastasis | 0.99 | 0.99 | 0.99 | 1.0 | 1.0 | * | 0.95 | 0.99 |
Liver metastasis | 0.99 | 1.0 | 0.99 | 1.0 | 0.97 | 0.98 | 1.0 | 1.0 |
Lung metastasis | 0.98 | 0.99 | 0.99 | 0.97 | 0.98 | 0.97 | 0.98 | 0.99 |
Nodal metastasis | 0.98 | 0.99 | 0.97 | 0.98 | 0.96 | 0.99 | 0.99 | 0.97 |
Peritoneal metastasis | 0.99 | 0.99 | 0.99 | 1.0 | 0.96 | 1.0 | * | 0.81 |
Models trained on imaging reports for patients with all cancers except for the type under evaluation | ||||||||
Any cancer | † | 0.98 | 0.98 | 0.95 | 0.95 | 0.98 | 0.98 | 0.96 |
Progression | † | 0.94 | 0.96 | 0.96 | 0.92 | 0.96 | 0.95 | 0.94 |
Response | † | 0.98 | 0.99 | 0.96 | 0.97 | 0.94 | 0.96 | 0.99 |
Brain metastasis | † | 0.97 | 1.0 | 0.99 | 1.0 | 0.98 | 0.99 | * |
Bone metastasis | † | 0.97 | 0.98 | 0.98 | 0.99 | 0.98 | 0.99 | 1.0 |
Adrenal metastasis | † | 0.99 | 0.99 | 0.99 | 1.0 | * | 0.94 | 0.99 |
Liver metastasis | † | 1.0 | 0.99 | 0.99 | 0.96 | 0.98 | 1.0 | 1.0 |
Lung metastasis | † | 0.99 | 0.99 | 0.94 | 0.97 | 0.97 | 0.97 | 0.99 |
Nodal metastasis | † | 0.99 | 0.98 | 0.98 | 0.96 | 0.99 | 0.99 | 0.97 |
Peritoneal metastasis | † | 0.99 | 0.98 | 1.0 | 0.97 | 1.0 | * | 0.77 |
