Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Advertisement

Scientific Reports
  • View all journals
  • Search
  • My Account Login
  • Content Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • RSS feed
  1. nature
  2. scientific reports
  3. articles
  4. article
A unified multi modal transformer framework for breast cancer recurrence prediction and survival analysis
Download PDF
Download PDF
  • Article
  • Open access
  • Published: 11 February 2026

A unified multi modal transformer framework for breast cancer recurrence prediction and survival analysis

  • Saleem Malik1,
  • S. Gopal Krishna Patro2,
  • Ahmed Kateb Jumaah Al-Nussairi3,
  • Chandrakanta Mahanty4,
  • Mohamed Ghouse5,
  • Akila Thiyagarajan5,
  • Ahmed Adnan Hadi6,
  • Anwar Khan7,
  • Mohit Mittal8 &
  • …
  • Amanuel Zewude9 

Scientific Reports , Article number:  (2026) Cite this article

  • 452 Accesses

  • Metrics details

We are providing an unedited version of this manuscript to give early access to its findings. Before final publication, the manuscript will undergo further editing. Please note there may be errors present which affect the content, and all legal disclaimers apply.

Subjects

  • Cancer
  • Computational biology and bioinformatics
  • Oncology

Abstract

Breast cancer recurrence prediction is an important feature of post-treatment therapy, requiring accurate identification of both recurrence risk and time-to-event outcomes. In this paper, we offer a unified deep learning system that jointly performs survival analysis and multi-class recurrence classification, enabling full risk stratification for breast cancer patients. The proposed model includes a Transformer-based survival module to estimate time-until-recurrence, and an attention-guided classification module to differentiate between second primary cancer, low-risk, and high-risk recurrence instances. A multi-modal dataset comprising clinical, molecular, demographic, and lifestyle data is created from established sources like METABRIC, GSE2034, GSE2990, BCSC, and the Breast Cancer Coimbra dataset. The model uses cross-modal feature fusion, autoencoder-based dimensionality reduction, and attention-based feature attribution for applicability and accessibility. Experimental results show better accuracy, precision, recall, and F1-score of 99.12%, 98.75%, 99.08%, and 98.91%, outperforming standard machine learning and survival models. This unified paradigm gives doctors a powerful, interpretable tool for early intervention and personalized breast cancer treatment.

Data availability

The data that supports the findings of this study are available within the article.

Code availability

The source code for the proposed framework is publicly available at the following GitHub repository: https://github.com/saleem-saleem/Breast-Cancer-Multimodal-Transformer.

Abbreviations

AI:

Artificial intelligence

AE:

Autoencoder

BCSC:

Breast cancer surveillance consortium

BMI:

Body mass index

BRCA:

Breast cancer gene (BRCA1/BRCA2)

CI:

Confidence interval

CNN:

Convolutional neural network

C-index:

Concordance Index

CoxPH:

Cox proportional hazards

DBT:

Digital breast tomosynthesis

DL:

Deep learning

ER:

Estrogen receptor

FFDM:

Full-field digital mammography

F1:

F1-score (Harmonic mean of precision and recall)

GEO:

Gene expression omnibus

HER2:

Human epidermal growth factor receptor 2

H&E:

Hematoxylin and eosin

HR:

Hazard ratio

KM:

Kaplan–meier

LASSO:

Least absolute shrinkage and selection operator

MLP:

Multilayer perceptron

METABRIC:

Molecular taxonomy of breast cancer international consortium

MMT:

Multi-modal transformer

MRI:

Magnetic resonance imaging

PH:

Proportional hazards

ReLU:

Rectified linear unit

ROC:

Receiver operating characteristic

SHAP:

SHapley additive exPlanations

SMOTE:

Synthetic minority oversampling technique

TCGA:

The cancer genome atlas

TNBC:

Triple-negative breast cancer

TIL:

Tumor-infiltrating lymphocyte

VAE:

Variational autoencoder

ViT:

Vision transformer

λ₁, λ₂:

Task weighting coefficients for survival and classification losses

γ:

Temperature scaling parameter in contrastive loss

τ:

Temperature parameter in similarity normalization

αₘ:

Modality attention weight

ψ(x;θ):

Log-risk function parameterized by network weights

References

  1. Kulothungan, V. et al. Burden of female breast cancer in India: Estimates of YLDs, YLLs, and DALYs at national and subnational levels based on the national cancer registry programme. Breast Cancer Res. Treat. 205, 323–332. https://doi.org/10.1007/s10549-024-07264-3 (2024).

    Google Scholar 

  2. Soldato, D. et al. The Future of breast cancer research in the survivorship field. Oncol. Ther. 11, 199–229. https://doi.org/10.1007/s40487-023-00225-8 (2023).

    Google Scholar 

  3. Beiranvand, M., Akbari, A. & Akbari, M. E. Evaluation of recurrence and survival in multifocal versus unifocal breast cancer patients at a tertiary center: A case-control study. Cancer Treat. Res. Commun. 43, 100894. https://doi.org/10.1016/j.ctarc.2025.100894 (2025).

    Google Scholar 

  4. van Maaren, M. C. et al. The effectiveness of personalised surveillance and aftercare in breast cancer follow-up: A systematic review. Support Care Cancer 32, 323. https://doi.org/10.1007/s00520-024-08530-2 (2024).

    Google Scholar 

  5. Tizi, W. & Berrado, A. Machine learning for survival analysis in cancer research: A comparative study. Sci. Afr. 21, e01880. https://doi.org/10.1016/j.sciaf.2023.e01880 (2023).

    Google Scholar 

  6. Kumari, D. et al. Predicting breast cancer recurrence using deep learning. Discov. Appl. Sci. 7, 113. https://doi.org/10.1007/s42452-025-06512-5 (2025).

    Google Scholar 

  7. Zhang, X. et al. Adaptive transformer modelling of density function for nonparametric survival analysis. Mach. Learn. 114, 31. https://doi.org/10.1007/s10994-024-06686-w (2025).

    Google Scholar 

  8. Wang, Z. et al. Surformer: An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images. Comput. Methods Programs Biomed. 241, 107733. https://doi.org/10.1016/j.cmpb.2023.10773 (2023).

    Google Scholar 

  9. Chen, X. et al. Artificial intelligence and multimodal data fusion for smart healthcare: Topic modeling and bibliometrics. Artif. Intell. Rev. 57, 91. https://doi.org/10.1007/s10462-024-10712-7 (2024).

    Google Scholar 

  10. https://www.kaggle.com/datasets/raghadalharbi/breast-cancer-gene-expression-profiles-metabric

  11. https://www.kaggle.com/datasets/saurabhshahane/gene-expression-profiles-of-breast-cancer

  12. https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE2990

  13. https://www.kaggle.com/datasets/sabahesaraki/bcsc-digital-mammography-dataset

  14. https://archive.ics.uci.edu/dataset/451/breast+cancer+coimbra

  15. Islam, T. et al. Predictive modeling for breast cancer classification in the context of Bangladeshi patients by use of machine learning approach with explainable AI. Sci. Rep. 14, 8487. https://doi.org/10.1038/s41598-024-57740-5 (2024).

    Google Scholar 

  16. Silveira, J. A., da Silva, A. R. & de Lima, M. Z. T. Harnessing artificial intelligence for predicting breast cancer recurrence: A systematic review of clinical and imaging data. Discov. Onc. 16, 135. https://doi.org/10.1007/s12672-025-01908-6 (2025).

    Google Scholar 

  17. Azeroual, S. et al. Predicting disease recurrence in breast cancer patients using machine learning models with clinical and radiomic characteristics: A retrospective study. J. Egypt. Natl. Canc. Inst. 36, 20. https://doi.org/10.1186/s43046-024-00222-6 (2024).

    Google Scholar 

  18. Vazifehdan, M., Moattar, M. H. & Jalali, M. hybrid Bayesian network and tensor factorization approach for missing value imputation to improve breast cancer recurrence prediction. J. King Saud Univ. – Comput. Inform. Sci. 31(2), 175–184. https://doi.org/10.1016/j.jksuci.2018.01.002 (2019A).

    Google Scholar 

  19. Terven, J. et al. A comprehensive survey of loss functions and metrics in deep learning. Artif. Intell. Rev. 58, 195. https://doi.org/10.1007/s10462-025-11198-7 (2025).

    Google Scholar 

  20. Jiang, S., Suriawinata, A. A. & Hassanpour, S. MHAttnSurv: Multi-head attention for survival prediction using whole-slide pathology images. Comput. Biol. Med. 158, 106883. https://doi.org/10.1016/j.compbiomed.2023.106883 (2023).

    Google Scholar 

  21. Zadeh Shirazi, A. et al. DeepSurvNet: Deep survival convolutional network for brain cancer survival rate classification based on histopathological images. Med. Biol. Eng. Comput. 58, 1031–1045. https://doi.org/10.1007/s11517-020-02147-3 (2020).

    Google Scholar 

  22. Mienye, I. D. et al. A survey of explainable artificial intelligence in healthcare: Concepts, applications, and challenges. Inform. Med. Unlocked 51, 101587. https://doi.org/10.1016/j.imu.2024.101587 (2024).

    Google Scholar 

  23. Mienye, I. D. & Swart, T. G. Deep autoencoder neural networks: A comprehensive review and new perspectives. Arch. Computat. Methods Eng. https://doi.org/10.1007/s11831-025-10260-5 (2025).

    Google Scholar 

  24. Schouten, D. et al. Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications. Med. Image Anal. 105, 103621. https://doi.org/10.1016/j.media.2025.103621 (2025).

    Google Scholar 

  25. Sallauka, R. et al. Scoping review: (Bio)markers for the prognostication of breast cancer recurrence. BMC Cancer 25, 1081. https://doi.org/10.1186/s12885-025-14515-z (2025).

    Google Scholar 

  26. Jannusch, K. et al. Prediction of therapy response of breast cancer patients with machine learning based on clinical data and imaging data derived from breast [18F]FDG-PET/MRI. Eur. J. Nucl. Med. Mol. Imaging 51, 1451–1461. https://doi.org/10.1007/s00259-023-06513-9 (2024).

    Google Scholar 

  27. Salvi, M. et al. Multi-modality approaches for medical support systems: A systematic review of the last decade. Inform. Fusion 103, 102134. https://doi.org/10.1016/j.inffus.2023.102134 (2024).

    Google Scholar 

  28. Luo, H. et al. Multimodal multi-instance evidence fusion neural networks for cancer survival prediction. Sci. Rep. 15, 10470. https://doi.org/10.1038/s41598-025-93770-3 (2025).

    Google Scholar 

  29. Javed, H., El-Sappagh, S. & Abuhmed, T. Robustness in deep learning models for medical diagnostics: security and adversarial challenges towards robust AI applications. Artif. Intell. Rev. 58, 12. https://doi.org/10.1007/s10462-024-11005-9 (2025).

    Google Scholar 

  30. Nazir, S., Dickson, D. M. & Akram, M. U. Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks. Comput. Biol. Med. 156, 106668. https://doi.org/10.1016/j.compbiomed.2023.106668 (2023).

    Google Scholar 

  31. Aghajanpour, S. et al. Utilizing machine learning for predicting drug release from polymeric drug delivery systems. Comput. Biol. Med. 188, 109756. https://doi.org/10.1016/j.compbiomed.2025.10975 (2025).

    Google Scholar 

  32. Senan, E. M., Alsaade, F. W., AlMashhadani, M. I. A., Theyazn, H. H. & Al-Adhaileh, M. H. Classification of histopathological images for early detection of breast cancer using deep learning. J. Appl. Sci. Eng. 24(3), 323–329 (2021).

    Google Scholar 

  33. Hameed, Z., Zahia, S., Garcia-Zapirain, B., Javier Aguirre, J. & María Vanegas, A. Breast cancer histopathology image classification using an ensemble of deep learning models. Sensors 20(16), 4373. https://doi.org/10.3390/s20164373 (2020).

    Google Scholar 

  34. Alom, M. R. et al. An explainable AI-driven deep neural network for accurate breast cancer detection from histopathological and ultrasound images. Sci. Rep. 15, 17531. https://doi.org/10.1038/s41598-025-97718-5 (2025).

    Google Scholar 

  35. Sánchez-Cauce, R., Pérez-Martín, J. & Luque, M. Multi-input convolutional neural network for breast cancer detection using thermal images and clinical data. Comput. Methods Programs Biomed. 204, 106045. https://doi.org/10.1016/j.cmpb.2021.106045 (2021).

    Google Scholar 

  36. Asadi, B. & Memon, Q. Efficient breast cancer detection via cascade deep learning network. Int. J. Intell. Netw. 4, 46–52. https://doi.org/10.1016/j.ijin.2023.02.001 (2023).

    Google Scholar 

  37. Sohail, A., Khan, A., Nisar, H., Tabassum, S. & Zameer, A. Mitotic nuclei analysis in breast cancer histopathology images using deep ensemble classifier. Med. Image Anal. 72, 102121. https://doi.org/10.1016/j.media.2021.10212 (2021).

    Google Scholar 

  38. Guo, D. et al. Classify breast cancer pathological tissue images using multi-scale bar convolution pooling structure with patch attention. Biomed. Signal Process. Control 96, 106607. https://doi.org/10.1016/j.bspc.2024.106607 (2024).

    Google Scholar 

  39. Saha, M. et al. Breast-NET: A lightweight DCNN model for breast cancer detection and grading using histological samples. Neural Comput. Applic. 36, 20067–20087. https://doi.org/10.1007/s00521-024-10298-9 (2024).

    Google Scholar 

  40. Ibraheem, A. M., Rahouma, K. H. & Hamed, H. F. A. 3PCNNB-Net: Three parallel CNN branches for breast cancer classification through histopathological images. J. Med. Biol. Eng. 41, 494–503. https://doi.org/10.1007/s40846-021-00620-4 (2021).

    Google Scholar 

  41. Wang, P. et al. Automatic classification of breast cancer histopathological images based on deep feature fusion and enhanced routing. Biomed. Signal Process. Control 65, 102341. https://doi.org/10.1016/j.bspc.2020.102341 (2021).

    Google Scholar 

  42. Huang, Z. et al. A novel nomogram for predicting locoregional recurrence risk in breast cancer patients treated with neoadjuvant chemotherapy and mastectomy. Radiother. Oncol. 161, 191–197. https://doi.org/10.1016/j.radonc.2021.06.015 (2021).

    Google Scholar 

  43. Huang, H. H. & Liang, Y. A novel cox proportional hazards model for high-dimensional genomic data in cancer prognosis. IEEE/ACM Trans. Comput. Biol. Bioinform. 18(5), 1821–1830. https://doi.org/10.1109/TCBB.2019.2961667 (2021).

    Google Scholar 

  44. B. U. Maheswari, A. A, A. Avvaru, A. Tandon and R. P. de Prado. "Interpretable Machine Learning Model for Breast Cancer Prediction Using LIME and SHAP". 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Pune, India, pp. 1–6, https://doi.org/10.1109/I2CT61223.2024.10543965. (2024).

  45. Zhong, Z. et al. Identification and validation of a novel 16-gene prognostic signature for patients with breast cancer. Sci. Rep. 12, 12349. https://doi.org/10.1038/s41598-022-16575-8 (2022).

    Google Scholar 

  46. Nemade, V. & Fegade, V. machine learning techniques for breast cancer prediction. Procedia Comput. Sci. 218, 1314–1320. https://doi.org/10.1016/j.procs.2023.01.110 (2023).

    Google Scholar 

  47. Emam, M. M., Houssein, E. H., Samee, N. A., Alohali, M. A. & Hosney, M. E. Breast cancer diagnosis using optimized deep convolutional neural network based on transfer learning technique and improved Coati optimization algorithm. Expert Syst. Appl. 255, 124581. https://doi.org/10.1016/j.eswa.2024.124581 (2024).

    Google Scholar 

  48. Noman, S. M. et al. Leveraging survival analysis and machine learning for accurate prediction of breast cancer recurrence and metastasis. Sci. Rep. 15, 3728. https://doi.org/10.1038/s41598-025-87622-3 (2025).

    Google Scholar 

  49. Luo, H., Huang, J., Ju, H., Zhou, T., & Ding, W. M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction. arXiv preprint arXiv:2408.04170. (2024).

  50. Lv, Z., Lin, Y., Yan, R., Wang, Y. & Zhang, F. TransSurv: Transformer-based survival analysis model integrating histopathological images and genomic data for colorectal cancer. IEEE/ACM Trans. Comput. Biol. Bioinform. 20(6), 3411–3420. https://doi.org/10.1109/TCBB.2022.3199244 (2022).

    Google Scholar 

  51. Ge, R., Hu, X., Huang, R., Jia, G., Wang, Y., Gu, R., Li, Y. TTMFN: Two-stream Transformer-based Multimodal Fusion Network for Survival Prediction. arXiv preprint arXiv:2311.07033. (2023).

  52. Gomaa, A. et al. Comprehensive multimodal deep learning survival prediction enabled by a transformer architecture: A multicenter study in glioblastoma. Neuro-Oncol. Adv. 6(1), vdae122 (2024).

    Google Scholar 

  53. Maigari, A., XinYing, C. & Zainol, Z. Multimodal deep learning breast cancer prognosis models: Narrative review on multimodal architectures and concatenation approaches. J. Med. Artif. Intell. https://doi.org/10.21037/jmai-24-146 (2025).

    Google Scholar 

  54. Othman, N. A., Abdel-Fattah, M. A. & Ali, A. T. A hybrid deep learning framework with decision-level fusion for breast cancer survival prediction. Big Data Cogn. Comput. 7(1), 50. https://doi.org/10.3390/bdcc7010050 (2023).

    Google Scholar 

  55. Witowski, J., Zeng, K. G., Cappadona, J., Elayoubi, J., Choucair, K., Chiru, E. D., Geras, K. J. Multi-modal AI for comprehensive breast cancer prognostication. arXiv preprint arXiv:2410.21256. (2024).

  56. Al-Hejri, A. M. et al. A hybrid explainable federated-based vision transformer framework for breast cancer prediction via risk factors. Sci. Rep. 15, 18453. https://doi.org/10.1038/s41598-025-96527-0 (2025).

    Google Scholar 

  57. Al-Hejri, A. M. et al. ETECADx: Ensemble self-attention transformer encoder for breast cancer diagnosis using full-field digital X-ray breast images. Diagnostics 13(1), 89. https://doi.org/10.3390/diagnostics13010089 (2023).

    Google Scholar 

  58. Park, J., Witowski, J., Xu, Y., Trivedi, H., Gichoya, J. A multi-modal AI system for screening mammography: Integrating 2D and 3D imaging to improve breast cancer detection in a prospective clinical study. arXiv preprint arXiv:2504.05636. https://arxiv.org/abs/2504.05636 (2025).

  59. Witowski, J., Zeng, K. G., Cappadona, J., Elayoubi, J. Multi-modal AI for comprehensive breast cancer prognostication. arXiv preprint arXiv:2410.21256. https://arxiv.org/abs/2410.21256 (2024).

  60. Shen, Y., Park, J., Yeung, F., Goldberg, E. Leveraging transformers to improve breast cancer classification and risk assessment with multi-modal and longitudinal data. arXiv preprint arXiv:2311.03217. https://arxiv.org/abs/2311.03217 (2023).

  61. Konz, N. et al. A competition, benchmark, code, and data for using artificial intelligence to detect lesions in digital breast tomosynthesis. JAMA Netw. Open 6(2), e230524. https://doi.org/10.1001/jamanetworkopen.2023.0524 (2023).

    Google Scholar 

  62. Bai, J., Posner, R., Wang, T., Yang, C. & Nabavi, S. Applying deep learning in digital breast tomosynthesis for automatic breast cancer detection: A review. Med. Image Anal. 71, 102049. https://doi.org/10.1016/j.media.2021.102049 (2021).

    Google Scholar 

  63. Abdelhafiz, D., Yang, C., Ammar, R., Nabavi, S. & Wang, T. Deep convolutional neural networks for mammography: Advances, challenges and applications. BMC Bioinformatics 20, 281. https://doi.org/10.1186/s12859-019-2823-6 (2019).

    Google Scholar 

  64. Abdelhafiz, D., Yang, C., Wang, T., Ammar, R. & Nabavi, S. Convolutional neural network for automated mass segmentation in mammography. BMC Bioinformatics 21, 52. https://doi.org/10.1186/s12859-020-3371-7 (2020).

    Google Scholar 

  65. Bai, J., Jin, A., Wang, T., Yang, C. & Nabavi, S. Feature fusion Siamese network for breast cancer detection comparing current and prior mammograms. Med. Phys. 49(6), 3654–3669. https://doi.org/10.1002/mp.15594 (2022).

    Google Scholar 

  66. Bai, J., Jin, A., Adams, M., Yang, C. & Nabavi, S. Unsupervised feature correlation model to predict breast abnormal variation maps in longitudinal mammograms. Comput. Med. Imaging Graph. 113, 102341. https://doi.org/10.1016/j.compmedimag.2024.102341 (2024).

    Google Scholar 

Download references

Acknowledgements

The authors extend their appreciation to the Deanship of Research and Graduate Studies at King Khalid University for funding this work through Small Research Project under grant number RGP1/141/46.

Funding

The authors extend their appreciation to the Deanship of Research and Graduate Studies at King Khalid University for funding this work through Small Research Project under grant number RGP1/141/46.

Author information

Authors and Affiliations

  1. CSE Department, P A College of Engineering, Mangalore, 574153, India

    Saleem Malik

  2. Department of Computer Science, Sreenidhi University, Hyderabad, Telangana, India

    S. Gopal Krishna Patro

  3. Dean of the Technical Engineering College, University of Manara, Maysan, Iraq

    Ahmed Kateb Jumaah Al-Nussairi

  4. Department of Computer Science & Engineering, GITAM Deemed to Be University, Visakhapatnam, 530045, India

    Chandrakanta Mahanty

  5. Department of Computer Science, College of Computer Science, King Khalid University, Abha, Kingdom of Saudi Arabia

    Mohamed Ghouse & Akila Thiyagarajan

  6. Artificial Intelligence Sciences Department, Al-Mustaqbal University, College of Sciences, 51001, Al Hillah, Babil, Iraq

    Ahmed Adnan Hadi

  7. Department of Electronics & Communication Engineering, Chandigarh University, Mohali, Punjab, India

    Anwar Khan

  8. Department of Data Science, Galgotias College of Engineering and Technology, Greater Noida, 201310, India

    Mohit Mittal

  9. School of Informatics and Computer Science, Dilla University, Po. Box 419, Dilla, Ethiopia

    Amanuel Zewude

Authors
  1. Saleem Malik
    View author publications

    Search author on:PubMed Google Scholar

  2. S. Gopal Krishna Patro
    View author publications

    Search author on:PubMed Google Scholar

  3. Ahmed Kateb Jumaah Al-Nussairi
    View author publications

    Search author on:PubMed Google Scholar

  4. Chandrakanta Mahanty
    View author publications

    Search author on:PubMed Google Scholar

  5. Mohamed Ghouse
    View author publications

    Search author on:PubMed Google Scholar

  6. Akila Thiyagarajan
    View author publications

    Search author on:PubMed Google Scholar

  7. Ahmed Adnan Hadi
    View author publications

    Search author on:PubMed Google Scholar

  8. Anwar Khan
    View author publications

    Search author on:PubMed Google Scholar

  9. Mohit Mittal
    View author publications

    Search author on:PubMed Google Scholar

  10. Amanuel Zewude
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Saleem Malik: Methodology; Software; Model development; Validation; Writing – review & editing. S. Gopal Krishna Patro: Formal analysis; Statistical analysis; Survival analysis, methodology; Validation; Writing – review & editing. Ahmed Kateb Jumaah Al-Nussairi: Clinical expertise; Domain validation; Interpretation of oncology-related results; Writing – review & editing. Chandrakanta Mahanty: Data curation; Feature engineering; Comparative analysis; Visualization; Writing – review & editing. Mohamed Ghouse: Software; Model training; Hyperparameter optimization; Experimental evaluation; Writing – review & editing. Akila Thiyagarajan: Data preprocessing; Ablation studies; Experimental support; Manuscript formatting and compliance; Writing – review & editing. Ahmed Adnan Hadi: Software implementation; Code verification; Reproducibility checks; Supplementary material preparation; Writing – review & editing. Anwar Khan: Literature review; related work synthesis; Conceptual alignment with existing oncology studies; Writing – review & editing. Mohit Mittal: Results analysis, Validation; Cross-validation design; Response to reviewer comments; Writing – review & editing. Amanuel Zewude: Conceptualization; Supervision; Project administration; multi-modal data integration, experimental workflow and submission correspondence. All authors have read and approved the final manuscript and agree to be accountable for all aspects of the work.

Corresponding authors

Correspondence to Saleem Malik or Amanuel Zewude.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary Information.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malik, S., Patro, S.G.K., Al-Nussairi, A.K.J. et al. A unified multi modal transformer framework for breast cancer recurrence prediction and survival analysis. Sci Rep (2026). https://doi.org/10.1038/s41598-026-37046-4

Download citation

  • Received: 25 August 2025

  • Accepted: 19 January 2026

  • Published: 11 February 2026

  • DOI: https://doi.org/10.1038/s41598-026-37046-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Breast cancer prediction
  • Deep learning
  • Survival analysis
Download PDF

Advertisement

Explore content

  • Research articles
  • News & Comment
  • Collections
  • Subjects
  • Follow us on Facebook
  • Follow us on X
  • Sign up for alerts
  • RSS feed

About the journal

  • About Scientific Reports
  • Contact
  • Journal policies
  • Guide to referees
  • Calls for Papers
  • Editor's Choice
  • Journal highlights
  • Open Access Fees and Funding

Publish with us

  • For authors
  • Language editing services
  • Open access funding
  • Submit manuscript

Search

Advanced search

Quick links

  • Explore articles by subject
  • Find a job
  • Guide to authors
  • Editorial policies

Scientific Reports (Sci Rep)

ISSN 2045-2322 (online)

nature.com sitemap

About Nature Portfolio

  • About us
  • Press releases
  • Press office
  • Contact us

Discover content

  • Journals A-Z
  • Articles by subject
  • protocols.io
  • Nature Index

Publishing policies

  • Nature portfolio policies
  • Open access

Author & Researcher services

  • Reprints & permissions
  • Research data
  • Language editing
  • Scientific editing
  • Nature Masterclasses
  • Research Solutions

Libraries & institutions

  • Librarian service & tools
  • Librarian portal
  • Open research
  • Recommend to library

Advertising & partnerships

  • Advertising
  • Partnerships & Services
  • Media kits
  • Branded content

Professional development

  • Nature Awards
  • Nature Careers
  • Nature Conferences

Regional websites

  • Nature Africa
  • Nature China
  • Nature India
  • Nature Japan
  • Nature Middle East
  • Privacy Policy
  • Use of cookies
  • Legal notice
  • Accessibility statement
  • Terms & Conditions
  • Your US state privacy rights
Springer Nature

© 2026 Springer Nature Limited

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer