Fig. 2: Multi-label model performance for 6–12 month predictions before pancreatic PC diagnosis.
From: Enhancing EHR-based pancreatic cancer prediction with LLM-derived embeddings
Using CUMC data, GPT embeddings demonstrated the best performance among all embedding types evaluated (Baseline, RGCN, GPT, and Mistral embeddings). GPT embeddings also consistently improved prediction performance in CSMC data. Larger discrepancies in prediction performance between CUMC and CSMC were observed when using all available data (AUROC 0.673 vs. 0.858), but these differences significantly diminished after excluding data from the 0–3 months prior to diagnosis, due to substantial improvement in the CUMC model (AUROC 0.819 vs. 0.893; Table 1). This suggests a more pronounced data leakage effect in the CUMC dataset, potentially driven by control group characteristics that resemble early PC features.
