ABSTRACT
We aimed to establish a robust vision-language model (“Glio-LLaMA-Vision”) for molecular status prediction and radiology report generation (RRG) in adult-type diffuse gliomas. Multiparametric MRI data and paired radiology reports from 1001 patients with adult-type diffuse gliomas were included in the institutional training set. A vision-language model, Glio-LLaMA-Vision, was developed from LLaMA 3.1 pre-trained on 2.79 million biomedical image-text pairs from PubMed Central and further fine-tuned from the institutional training set. The performance was validated in 100 patients and 75 patients with paired MRI-radiology reports from an institutional validation set and another tertiary institution (AMC), and in 170 and 477 patients with MRI from TCGA and UCSF datasets, respectively. In terms of IDH mutation status prediction, Glio-LLaMA-Vision showed AUCs ranging from 0.85–0.95 in the internal validation and external datasets. In terms of RRG, the BLEU-1 and ROUGE-L scores were 0.50 and 0.49 in the internal validation, respectively, and 0.32 and 0.36 on the AMC dataset, respectively. Overall, 37.8% of generated reports were considered superior or equal to the original reports, while 91.0% of generated reports were considered clinically acceptable by neuroradiologists. In conclusion, Glio-LLaMA-Vision demonstrates promising performance in molecular status prediction and RRG in adult-type diffuse gliomas, showing potential for clinical assistance.
Data availability
The datasets analyzed during the current study are not publicly available due to institutional and ethical restrictions but are available from the corresponding author on reasonable request. Access to the data will be provided after review and approval by all authors and in compliance with applicable institutional policies and ethical guidelines.
Code availability
A valid OSI-approved open-source license, the MIT License, is applied to our code. The code repository includes a clear LICENSE file specifying that the code is released under the MIT License. Code is publicly available at (https://github.com/myeongkyunkang/Glio-LLaMA-Vision). Other information is available from the corresponding author upon request.
References
Ostrom, Q. T. et al. CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2016-2020. Neuro Oncol. 25, iv1–iv99 (2023).
Louis, D. N. et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro Oncol. 23, 1231–1251 (2021).
Brat, D. J. et al. Comprehensive, integrative genomic analysis of diffuse lower-grade gliomas. N. Engl. J. Med. 372, 2481–2498 (2015).
Turkalp, Z., Karamchandani, J. & Das, S. IDH mutation in glioma: new insights and promises for the future. JAMA Neurol. 71, 1319–1325 (2014).
Sarkar, C. et al. Resource availability for CNS tumor diagnostics in the Asian Oceanian region: a survey by the Asian Oceanian Society of Neuropathology committee for Adapting Diagnostic Approaches for Practical Taxonomy in Resource-Restrained Regions (AOSNP-ADAPTR). Brain Pathol. e13329, https://doi.org/10.1111/bpa.13329 (2025).
Vollmuth, P. et al. A Radiologist’s Guide to IDH-Wildtype Glioblastoma for Efficient Communication With Clinicians: PartI-Essential Information on Preoperative and Immediate Postoperative Imaging. Korean J. Radiol. 26, 246–248 (2025).
Park, Y. W. et al. The 2021 WHO classification for gliomas and implications on imaging diagnosis: part 1-key points of the fifth edition and summary of imaging findings on adult-type diffuse gliomas. J. Magn. Reson. Imaging 58, 677–689 (2023).
Smith-Bindman, R. et al. Trends in use of medical imaging in US health care systems and in Ontario, Canada, 2000-2016. Jama 322, 843–856 (2019).
Kim, K. et al. Updated primer on generative artificial intelligence and large language models in medical imaging for medical professionals. Korean J. Radiol. 25, 224–242 (2024).
Nam, Y. et al. Multimodal large language models in medical imaging: current state and future directions. Korean J. Radiol. 26, 900–923 (2025).
Faghani, S., Park, Y. W. & Park, J. E. Uncover This Tech Term: Large Vision-Language Models in Radiology. Korean J. Radiol. 27, 375–378 (2026).
Huang, W. et al. Enhancing representation in radiography-reports foundation model: a granular alignment algorithm using masked contrastive learning. Nat. Commun. 15, 7620 (2024).
Tanno, R. et al. Collaboration between clinicians and vision-language models in radiology report generation. Nat. Med. 31, 599–608 (2025).
Chen, Z. et al. A Vision-Language foundation model to enhance efficiency of chest x-ray interpretation. arXiv e-prints, arXiv: https://arxiv.org/abs/2401.12208 (2024).
Dubey, A. et al. The llama 3 herd of models. arXiv preprint arXiv: https://arxiv.org/abs/2407.21783 (2024).
Cluceru, J. et al. Improving the noninvasive classification of glioma genetic subtype with deep learning and diffusion-weighted imaging. Neuro Oncol. 24, 639–652 (2022).
van der Voort, S. R. et al. Combined molecular subtyping, grading, and segmentation of glioma using multi-task deep learning. Neuro Oncol. 25, 279–289 (2023).
Choi, Y. S. et al. Fully automated hybrid approach to predict the IDH mutation status of gliomas via deep learning and radiomics. Neuro Oncol. 23, 304–313 (2021).
Byeon, Y. et al. Interpretable multimodal transformer for prediction of molecular subtypes and grades in adult-type diffuse gliomas. npj Digit. Med. 8, 140 (2025).
Xiao, H. et al. A comprehensive survey of large language models and multimodal large language models in medicine. Inf. Fusion 117, 102888 (2025).
Li, C. et al. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Adv. Neural Inf. Process. Syst. 36, 28541–28564 (2023).
Tu, T. et al. Towards generalist biomedical AI. Nejm AI 1, AIoa2300138 (2024).
Hoopes, A., Butoi, V. I., Guttag, J. V. & Dalca, A. V. Voxelprompt: a vision-language agent for grounded medical image analysis. arXiv preprint arXiv: http://arxiv.org/abs/2410.08397 (2024).
Tak, D. et al. A foundation model for generalized brain MRI analysis. medRxiv, https://doi.org/10.1101/2024.12.02.24317992 (2024).
Gonzales, Ricardo A. et al. Metrics for Artificial Intelligence in Medicine: A Reference Resource. Radiol.: Artif. Intell. e260070 (2026).
Park, S. H. & Kim, N. Challenges and proposed additional considerations for medical device approval of large language models beyond conventional AI. Radiology 312, e241703 (2024).
Yi, P. H. et al. Best practices for the safe use of large language models and other generative AI in radiology. Radiology 316, e241516 (2025).
Faghani, S. et al. Quantifying uncertainty in deep learning of radiologic images. Radiology 308, e222217 (2023).
Park, Y. W. et al. Differentiation of glioblastoma from solitary brain metastasis using deep ensembles: empirical estimation of uncertainty for clinical reliability. Comput. Methods Prog. Biomed. 254, 108288 (2024).
Louis, D. N. et al. cIMPACT-NOW update 1: not otherwise specified (NOS) and not elsewhere classified (NEC). Acta Neuropathol. 135, 481–484 (2018).
Gutman, D. A. et al. MR imaging predictors of molecular profile and survival: multi-institutional study of the TCGA glioblastoma data set. Radiology 267, 560–569 (2013).
Calabrese, E. et al. The University of California San Francisco preoperative diffuse glioma MRI dataset. Radio. Artif. Intell. 4, e220058 (2022).
Isensee, F. et al. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 40, 4952–4964 (2019).
Kickingereder, P. et al. Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol. 20, 728–740 (2019).
Zhang, S. et al. A multimodal biomedical foundation model trained from fifteen million image–text pairs. NEJM AI 2, AIoa2400640 (2025).
Yang, L. et al. Advancing multimodal medical capabilities of Gemini. arXiv preprint arXiv: http://arxiv.org/abs/2405.03162 (2024).
Hasani, A. M. et al. Evaluating the performance of generative pre-trained transformer-4 (GPT-4) in standardizing radiology reports. Eur. Radiol. 34, 3566–3574 (2024).
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2025-00516124, RS-2025-00515423, and RS-2025-00515536). This study was also supported by a research grant from Yonsei University College of Medicine (4-2022-0828).
Author information
Authors and Affiliations
Contributions
Y.W.P., K.H., Y.S., J.E.P., J.H.C., S.H.K., S.L., and S.S.A. collected and curated the data. Y.W.P., M.K., S.H.P., and S.S.A. conceptualized the study and developed the methodology. M.K. and H.R. developed the software. Y.W.P., M.K., H.R., K.H., Y.S., J.E.P., J.H.C., S.H.K., S.K., S.H.P., and S.S.A. performed the validation. Y.W.P., M.K., and H.R. performed the statistical analysis. S.H.P. and S.S.A. designed the study and supervised the research. Y.W.P. and M.K. wrote the original draft. All authors reviewed and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Park, Y.W., Kang, M., Ryu, H. et al. A robust vision language model for molecular status prediction and radiology report generation in adult-type diffuse gliomas. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-02581-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41746-026-02581-x
