Fig. 4

Performance of the Multi-Modal Model (MuMo). a–c Receiver operating characteristic (ROC) curves display MuMo’s performance in predicting treatment responses, distinguishing between non-responders and responders. These curves pertain to the validation and test sets of the anti-HER2 cohort and the validation set of the anti-HER2 combined immunotherapy cohort. d–f Kaplan–Meier (KM) curves depict Progression-Free Survival (PFS) based on MuMo predictions. These curves were derived from the validation and test sets of the anti-HER2 cohort and the validation set of the anti-HER2 combined immunotherapy cohort. In the Kaplan–Meier analysis, patients were categorized into high-risk (red line) and low-risk (blue line) groups using the Youden index. The log-rank (Mantel-Cox) test was used to determine statistical significance, with a two-sided P-value of < 0.05 set as significant. g–i Kaplan–Meier (KM) curves show Overall Survival (OS) based on MuMo predictions for the validation and test sets of the anti-HER2 cohort and the validation set of the anti-HER2 combined immunotherapy cohort. j Stability analysis of MuMo in treatment response prediction. Ablation studies evaluating the integration of radiological and pathological data (k), the impact of various integration modes (l), and the inclusion of clinical information within MuMo (m). Error bars represent the 95% confidence intervals (CI) for the AUC scores