Introduction

Cancer manifests across multiple biological scales, from molecular alterations and cellular morphology to tissue organization and clinical phenotype. Predictive models that rely on a single data modality fail to capture multiscale heterogeneity, limiting their ability to generalize our understanding across patient populations. Multimodal artificial intelligence (MMAI) approaches integrate information from diverse sources, including cancer multiomics, histopathology, clinical records, and others, enabling models to exploit biologically meaningful inter-scale relationships (Fig. 1)1.

Fig. 1: MMAI integrates diverse data sources to generate comprehensive insights.
figure 1

*Local data sources or other epidemiological data. AI artificial intelligence, MMAI multimodal artificial intelligence.

Full size image

MMAI approaches enhance predictive accuracy and robustness by contextualizing molecular features within anatomical and clinical frameworks, yielding a more comprehensive representation of disease. Such models are more likely to support mechanistically plausible inferences, improving interpretability and clinical relevance2,3. MMAI presents an opportunity to advance cancer management, from prevention and early, accurate diagnosis to prognosis, treatment selection, and outcome assessment (Fig. 2)1.

Fig. 2: In oncology, MMAI enhances the possibility of personalized care at each stage of the patient journey.
figure 2

AI artificial intelligence, MMAI multimodal artificial intelligence.

Full size image

As artificial intelligence (AI) models become increasingly integral to the cancer care continuum, rigorous evaluation of their transformative potential, implementation barriers, and future directions is imperative. Here, we highlight clinical applications of MMAI, including examples of pioneering clinical research in personalized oncology care, as well as ethical and practical considerations for widespread implementation of MMAI.

Prevention and early detection

Personalized prevention strategies

AI facilitates personalized disease prevention by identifying high-risk individuals and recommending tailored lifestyle measures, from neurodegenerative and infectious diseases to oncology4. For example, epidemiological MMAI models from the UK Biobank integrated clinical, lifestyle, and polygenic risk variables to identify cardiovascular disease among patients with non-alcoholic fatty liver disease (area under the receiver operating curve [ROC–AUC] 0.85)5. Additionally, MMAI models integrating clinical and demographic variables identified patients with onco-hematological diseases who are likely to develop severe COVID-19, offering an opportunity for additional prevention strategies6.

These examples highlight the potential of epidemiological MMAI models to analyze population health data to predict disease risk factors and suggest targeted patient monitoring and chemoprevention strategies.

Screening and risk stratification

MMAI-based predictive capabilities have enabled targeted screening and risk stratification, allowing earlier intervention that may improve patient outcomes. MMAI algorithms can stratify cancer risk at various stages of the patient care pathway. A study of >5000 patients showed machine learning models, using clinical metadata, mammography, and trimodal ultrasound, were similar or better at predicting breast cancer risk compared with pathologist-level assessments7. Models such as Sybil AI demonstrated up to 0.92 ROC–AUC in predicting lung cancer risk from low-dose computed tomography (CT) scans, and could be incorporated into CT screening programs without disruption to current clinical workflows8. Given their potential, multimodal screening programs are needed to ensure external validity and equitable applicability. Project MONAI (Medical Open Network for AI), co-founded by Nvidia, is an open-source, PyTorch-based framework providing a comprehensive suite of AI tools and pre-trained models for medical imaging applications9. In breast cancer screening, MONAI-based models enable precise delineation of the breast area in digital mammograms, improving both accuracy and efficiency of breast cancer screening programs10. For ovarian cancer, deep learning models developed with MONAI enhance diagnostic accuracy on CT and magnetic resonance imaging scans11. In lung cancer, MONAI facilitates the integration of radiomics and patient demographic data within deep learning models, leading to improved risk assessment and screening outcome accuracy compared with Lung Imaging Reporting and Data System (Lung-RADS) classification12.

AI-enhanced early detection

Deep learning models can enhance the sensitivity of cancer screening and early detection programs by detecting subtle tumor features on medical imaging. In a meta-analysis, AI-guided colonoscopy increased adenoma detection rate by ~10%13, and emerging AI-assisted liquid biopsy models are able to identify cell-free DNA (cfDNA), potentially revolutionizing non-invasive early detection14,15.

Following early detection, timely and accurate diagnosis are needed to optimize treatment plans; integration of AI tools into clinical workflows is therefore being explored in primary care for early identification of patients with lung and ovarian cancers16.

Diagnosis and prognosis

AI diagnosis in pathology and radiology

In digital pathology, numerous AI-assisted diagnostic approaches have been published, a meta-analysis of which achieved 96.3% sensitivity and 93.3% specificity across common tumor-type classifiers17. Furthermore, lightweight architectures, such as ShuffleNet, can infer genomic alterations directly from histology slides (ROC–AUC 0.89), reducing turnaround time and cost of targeted sequencing across solid tumors18.

AI-powered imaging systems enhance tumor detection, lesion characterization, and disease staging. In radiology, multiple US Food and Drug Administration (FDA)-approved AI applications now assist in identifying breast lesions on mammograms and pulmonary nodules on CT scans19,20. When combined with clinical metadata in multimodal models, prognostic accuracy and survival prediction improve further, as demonstrated by F-18 fluorodeoxyglucose positron emission tomography images in non-small cell lung cancer (NSCLC)21.

Prognosis and outcome prediction

MMAI can integrate imaging, histology, genomics, and other biomarker and clinical data to forecast progression and therapy response. Stanford’s MUSK, a transformer-based AI model, achieved improved accuracy for melanoma relapse and immunotherapy response prediction (ROC–AUC 0.833 for 5-year relapse prediction) compared with existing unimodal approaches22, and Pathomic Fusion, a multimodal fusion strategy combining histology and genomics in glioma and clear-cell renal-cell carcinoma datasets, outperformed the World Health Organization 2021 classification for risk stratification23. Furthermore, a pan-tumor analysis of 15,726 patients combined multimodal real-world data and explainable AI to identify 114 key markers across 38 solid tumors, which were subsequently validated in an external lung cancer cohort24.

AstraZeneca’s ABACO, a pilot real-world evidence (RWE) platform utilizing MMAI, applies similar principles at scale to identify predictive biomarkers for targeted treatment selection, optimize therapy response predictions, and improve patient stratification in patients with hormone receptor–positive (HR+) metastatic breast cancer25. Another example is the TRIDENT machine learning multimodal model, which integrates radiomics, digital pathology, and genomics data from the Phase 3 POSEIDON study in patients with metastatic NSCLC26. These models yielded a patient signature in >50% of the population that would obtain optimal benefit from a particular treatment strategy (hazard ratio [HR] reduction: 0.88–0.56, non-squamous histology population; 0.88–0.75, intention-to-treat population)27. These advances underscore the benefit of MMAI-powered models for refining precision medicine strategies beyond current standard-of-care frameworks.

Personalized treatment and patient management

AI-driven precision oncology

In precision oncology, treatment selection is compounded by numerous small molecularly defined subgroups and an expanding arsenal of targeted therapies. Characterizing patients within these subgroups requires the integration of high-dimensional data, including hundreds of variables per patient. For instance, molecular diagnostics may assess gene mutations, copy number variations, and expression levels, while imaging data provide spatial and morphological context that reflect tumor heterogeneity and microenvironment28,29. The complexity of these multimodal datasets renders traditional decision-making tools insufficient. Thus, MMAI models can support clinicians by providing personalized treatment recommendations that consider the intricate interplay of diverse data types and therapeutic options.

Benchmarking efforts demonstrate the promise of MMAI in this space. The Dialogue on Reverse Engineering Assessment and Methods (DREAM) drug sensitivity prediction challenge assessed the ability of AI models to predict drug response using multiomics data in breast cancer cell lines, revealing that multimodal approaches consistently outperform unimodal ones in predicting therapeutic outcomes30. In breast cancer, multimodal models based on the TransNEO, ARTemis, and PBCP studies showed that response to treatment is modulated by pre-treated tumor ecosystems31. An MMAI patient stratification model used data from five Phase 3 prostate cancer trials to predict long-term, clinically relevant outcomes, with 9.2–14.6% relative improvement compared with a validation set based on National Cancer Center Network risk stratification32. Furthermore, in metastatic NSCLC, the TRIDENT initiative identified mutational signatures of patients likely to benefit from combination treatments27.

Remote monitoring and AI-assisted symptom management

AI-powered telehealth platforms and wearable sensors may facilitate real-time patient monitoring, enabling early detection of treatment-related complications, and AI-driven chatbots can alleviate physician workload, improve patient engagement, and reduce emergency hospital visits33,34,35,36. For example, a UK pilot study using a chatbot approach to triage patients with gynecological malignancies receiving chemotherapy achieved an unscheduled emergency visit adjusted incident ratio rate of 0.3135.

AstraZeneca’s RWE platform, ABACO, incorporates MMAI into real-world continuous monitoring, linking treatment outcomes to dynamic AI-driven insights to enhance patient management25. Thus, through multimodal integration of remote patient monitoring and conventional data streams, the platform can capture complementary physiological and contextual information, thereby improving predictive performance and enabling more precise, timely clinical decision-making. This may enable oncologists to proactively adjust treatment and management plans specific to each patient, thereby minimizing adverse events.

Quality of life beyond systemic treatments

Ensuring maintenance or improvement of quality of life beyond systemic treatments is challenging to address. QUALITOP followed 1800 patients treated with immune-checkpoint inhibitors or chimeric antigen receptor (CAR-T) cells, using AI models to link adverse events with health-related quality of life scores37. The recent inclusion of electronic patient-reported outcomes offers a new modality that, once integrated into the multimodal workstream, can inform patient-centered care.

Drug development and clinical research implications

AI-accelerated drug development

AI-driven drug discovery platforms (e.g., BenevolentAI) analyze large-scale molecular datasets to identify promising drug candidates38. Although the sample size is limited, AI-designed molecules are now estimated to progress to clinical trials at twice the rate of traditionally developed drugs, and early reviews suggest a success rate of 80–90% for Phase 1 clinical trials, which is substantially higher than the industry standard39. Machine learning models could expedite target identification and lead optimization, thereby significantly reducing timelines for new oncology drug development.

Enhancing clinical trial efficiency

AI optimizes clinical trial recruitment by matching patients with studies based on tumor biomarkers, improving enrollment efficiency. Eligibility‑matching engines reduce manual screening time, while real-time adaptive randomization informed by MMAI analytics reallocates patients toward superior arms earlier40,41.

The MMAI ABACO framework may facilitate patient enrichment strategies, leading to more efficient and precise drug trials based on RWE25. The TRIDENT initiative has the ability to optimize biomarker-driven patient selection and synthetic control arms, which may reduce trial costs and accelerate approvals27. These AI-driven comparator cohorts (‘digital twin’) have the potential to reduce reliance on traditional randomized control groups and are validated in chronic graft-versus-host disease42; they are also being spearheaded in rare diseases and oncology43,44.

MMAI may offer an ethical and efficient alternative in oncology, particularly for targeted therapies in rare cancers and in supporting clinical trials where patient recruitment is challenging. Overall, by integrating the synthetic control arm methodology, the clinical trial process may be further streamlined, ensuring robust regulatory submissions and improved decision-making in drug development (Fig. 3).

Fig. 3
figure 3

Synthetic control arms use external patient data derived from real-world data or previous clinical trials to compare the effectiveness of an intervention, to streamline the clinical trial process.

Full size image

Boosting patient recruitment

Using AI to streamline patient recruitment into clinical trials has enabled physicians to assess and manage patients’ conditions faster. GatorTron, a large language model for electronic health records (EHRs), achieved state-of-the-art performance in early identification of patients with a potential diabetes diagnosis by extracting phenotypes for trial across 290 million clinical notes45. Trial Pathfinder, an AI algorithm developed using retrospective real-world data from 61,094 patients with advanced NSCLC, was created to simplify the exclusion and inclusion criteria of clinical trials. Evaluation of clinical trial criteria with Trial Pathfinder suggested that 17% more patients could qualify for second-line trials46.

Although challenges are expected in integrating MMAI in clinical practice, such models have great potential to enhance patient identification and support patient equity.

Benefits to the healthcare system

Data availability and curation

A persistent barrier to MMAI lies in fragmented data silos. Initiatives such as the EU Health Data Space Regulations and the American Society of Clinical Oncology’s CancerLinQ illustrate a trend toward ‘radical interoperability’ based on Fast Healthcare Interoperability Resources (FHIR) and Observational Medical Outcomes Partnership (OMOP) standards for large-scale analytics47,48. The TRIDENT initiative exemplifies how a well-curated Phase 3 dataset (~2.2 TB of imaging and omics [Colin, T. 2025; personal communication]) can fuel externally validated MMAI models27. Adequately regulated or federated learning frameworks may further allow algorithms to train on ‘distributed nodes’ without centralizing patient‑level data, thus preserving privacy while expanding sample diversity.

Treatment efficiency and cost-effectiveness

Escalating investment and risk-taking when developing new targeted drugs and immunotherapies make oncology one of the fastest-growing cost lines for health systems worldwide49,50. MMAI could mitigate this pressure by improving the fit between patient and therapy, shortening diagnostic turnaround times, and generating evidence that payers can use to structure value-based contracts (Fig. 4).

Fig. 4: Ways that multimodal AI can enhance patient outcomes and cost-effectiveness.
figure 4

AI artificial intelligence.

Full size image

Interoperable data result in system‑level gains

Deploying MMAI on harmonized datasets (e.g., FHIR-OMOP standards) means that radiology, digital pathology, genomic feeds, and others are analyzed together rather than in silos. A hospital‑wide return on investment (ROI) analysis of an AI radiology platform estimated a 5‑year ROI of 451% (791% when radiologist time savings were monetized), equivalent to 145 working days of imaging workflow saved. These savings accrued to providers through resource productivity and payers through improved care and potential reduction in litigations from missed diagnoses51.

Better diagnostic capability reduces overtreatment

A proof point is provided by genomic assays in node‑negative, HR+ breast cancer: the Oncotype DX Recurrence Score spared chemotherapy for low‑risk patients and, in a UK cost‑utility model, delivered an incremental gain of 0.17 quality-adjusted life years (QALYs) while reducing lifetime costs by £519 per patient52. MMAI platforms extend this principle by blending digital pathology with radiomics and clinical variables and identifying a respondent patient signature, as demonstrated by TRIDENT27, with the potential for improved QALY gains in high-benefiting patients.

Diagnostics and targeted treatment shift the cost‑effectiveness frontier

ctDNA represents an additional data layer that can be combined with clinicopathological features, digital pathology, and radiomic features to identify targeted interventions. A good example is ctDNA-guided adjuvant therapy in the DYNAMIC trial (455 patients with Stage II colon cancer), whereby a liquid biopsy minimal residual disease signal plus clinicopathological data led to a reduction in adjuvant chemotherapy use from 28% to 15% and achieved a similar 2‑year recurrence‑free survival (93.5% vs 92.4%)53. In practical terms, four patients needed to be tested to spare one from unnecessary chemotherapy and its toxicity.

The aforementioned strategies lead to superior health economic outcomes, meaning enhanced health results coupled with reduced costs. This underscores the potential for MMAI approaches to tackle the substantial challenges that healthcare systems encounter due to budget constraints, while striving to improve patient outcomes.

Payer engagement

MMAI outputs are machine-readable, version-controlled, and linked to outcomes registries and leveraged by frameworks, such as the ABACO RWE platform25. This facilitates:

  • Performance‑based agreements – payment triggers only when the algorithm‑predicted benefit is realized

  • Dynamic pricing – drug discounts tied to MMAI‑quantified responder fractions

  • Global portability – models can be federated; low‑ and middle‑income countries can adopt validated algorithms without exporting data, therefore broadening access while preserving sovereignty

Collectively, these mechanisms realign incentives toward maximizing health. Prospective medico‑economic evaluations are now a prerequisite for AI biomarkers to obtain broad reimbursement, ensuring that cost‑effectiveness is embedded from the outset54.

Health equity

We believe that consciously designed MMAI can help narrow global cancer care inequities. For example, AI-enabled telepathology services used in several sub‑Saharan countries demonstrated concordance rates of between 70% and 95% compared with traditional methods of diagnosis55. Implementation of AI-driven digital microscopy diagnostics for Papanicolaou tests to detect atypical cervical smears in resource-limited settings achieved high levels of sensitivity and specificity56, with potential to include other modalities and reduce time to treatment and outcomes. These examples are particularly relevant as they target a region with fewer than one pathologist for every 1,100,000 people57. Similar gains are reported for teleradiology networks in Southeast Asia, where AI tools for chest X-rays are mitigating radiologist shortages by enhancing diagnostic image analysis and triaging priority reporting by radiologists58.

With context‑appropriate infrastructure, AI models, and now MMAI, can further support health systems, especially in regions with limited resources. A challenge for health equity that needs addressing is the need for models to be trained in cohorts that are representative of the full population; otherwise, there is a risk of augmenting inequalities.

Challenges and future directions

AI adoption in oncology necessitates seamless integration between its development and its deployment into EHRs and clinician workflows. Ensuring transparency, explainability, regulatory compliance, and overcoming the ‘black box’ – or unknown – working mechanism by gaining clinician trust through education on AI technologies is paramount59. In the context of MMAI for clinical decision support systems (CDSS), the derivation of robust, minimal, and independent predictive signatures, rooted in biological plausibility, is indispensable for ensuring transparency, interpretability, clinical utility, and adoption.

Furthermore, ethical concerns surrounding data privacy, decentralized data sharing, and regulated/federated learning nodes, broader stakeholder involvement, and their governance demand urgently harmonized guidelines and security measures for proper scalability.

Data and regulatory framework, and stakeholder engagement

MMAI relies on extensive patient data; frameworks such as the General Data Protection Regulation and the Health Insurance Portability and Accountability Act60 focus on data privacy, ownership, and secondary use, ensuring the patient and the patient’s interests are respected. Although regulatory AI-specific guidance has been published, including the European Medicines Agency’s reflection paper on the use of AI in the medicinal product lifecycle61 and FDA’s Good Machine Learning Practice principles62, there is urgency to address scalability and geographic challenges, while facilitating a homogeneous global framework for multi-stakeholders to work toward a common goal63,64,65. We acknowledge the need for establishing collaborative platforms where researchers and industry experts, from healthcare providers to regulatory agencies, the pharmaceutical industry to governments, the technology industry and medical devices, patient groups, and policy makers, can all share findings, best practices, and case studies demonstrating AI’s benefits. Cross-stakeholder engagement is key for MMAI to thrive.

Federated learning

Federated learning is a transformative approach in healthcare, enabling the development of AI models across multiple institutions without the need to centralize sensitive patient data66,67.

Although federated learning has been explored for drug discovery, its benefits are comparatively greater and have materialized in MMAI applications for CDSS. In pharmaceutical research and development, federated learning faces additional challenges beyond privacy: safeguarding of proprietary information such as molecular structures, compound libraries, experimental protocols, and intellectual property. Consequently, the cost-benefit ratio of federated learning is likely less favorable in pharmaceutical research and development compared with in CDSS. Novel paradigms such as swarm learning, which decentralize both data and model orchestration through blockchain-based peer-to-peer governance, are beginning to overcome some of these challenges68. These decentralized architectures reduce reliance on a central server, mitigate single points of failure, and enhance trust and auditability — features particularly valuable in cross-institutional clinical collaborations68. Furthermore, hybrid models combining transfer learning with privacy-preserving techniques such as differential privacy, homomorphic encryption, or secure multiparty computation are expanding the frontier of what can be learned from distributed data69. These innovations make federated approaches not only more robust and secure but also more scalable and adaptable to the heterogeneity of real-world healthcare systems.

Bias management

AI can reveal latent biases but also perpetuate them if left unchecked. Prevention approaches include automated feature selection, adversarial debiasing, and cross-modal consistency checks. Although randomized clinical trials (RCTs) remain the gold standard, post-hoc analyses are often required to determine subgroups of patients with greater response rates and regulatory purposes. Subgroups are commonly determined based on mono- or bivariate analyses, with specific features sought to show a treatment-modifying effect70. Multivariate analyses are important to determine complex interdependencies of covariates that identify patients most likely to benefit from treatments; however, they can be time-consuming, cause delays in trial readouts, and lack the robustness needed for regulators and payers.

MMAI is best viewed as a complementary analytical layer that can generate hypotheses and support personalized subgroup analyses based on multivariate features, with further potential to identify minimal, independent features that are meaningful to treatment outcomes. A shift toward a priori inclusion of MMAI hypotheses within RCTs could transform future trial design, leading to more personalized trials without the need for post-hoc analyses and their associated perception of bias.

To be adequately assessed and incorporated into clinical practice, a new hierarchy of MMAI evidence modeling is needed (i.e., incorporating real-world data, synthetic controls, and continuous learning systems).

Conclusion

Advances in MMAI offer a unique opportunity to revolutionize oncology by enhancing early detection, refining diagnostics, optimizing trial efficiency, expediting drug development, and personalizing treatment strategies. Initiatives that harness insights from Phase 3 clinical trial data or that are powered by RWE exemplify how MMAI can translate into tangible clinical benefit. While challenges remain, including the necessity for a robust regulatory framework and a collaborative healthcare ecosystem, the potential to predict risks and outcomes from screening to treatment selection through the AI-driven integration of clinical and biological datasets is poised to fundamentally transform oncology and clinical care. This transformation promises considerable benefits for individual patient outcomes, marking a new era in precision medicine.