Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

Ocular toxicities associated with MEK/BRAF inhibitors: assessing the accuracy and completeness of large language models

Eye (2025)Cite this article

Subjects

Access through your institution
Buy or subscribe

Artificial intelligence (AI) is transforming healthcare by improving diagnostic accuracy and treatment planning [1]. Large language models (LLMs) such as ChatGPT, Bard, and Llama analyse extensive medical data and assist clinicians [2]. In ophthalmology, AI has been used to detect diabetic retinopathy and age-related macular degeneration [3, 4]. However, its role in evaluating ocular toxicities related to oncologic drugs remains unexplored. This study assesses the accuracy and completeness of LLM responses on MEK/BRAF inhibitor-related ocular toxicity questions.

Eight questions evaluated MEK/BRAF inhibitor definitions, mechanisms, ocular adverse effects, and management (Table 1). In November 2023, each question was entered into ChatGPT 4.0, Bard, and LLaMA 2 70B three times, each in a separate session without prior history to minimise memory bias. The first response generated by each model was used for rating to ensure objective and reproducible comparisons across models, and four ocular toxicity experts, blinded to the LLM source, assessed accuracy and completeness using a 6-point Likert scale. The Kruskal-Wallis test was used to compare the average ratings of the four experts across LLMs, followed by post-hoc pairwise Mann-Whitney U tests to identify significant differences. To compare questions addressing general properties of MEK/BRAF inhibitors (#1-2) with those examining ocular toxicity (#3-8), irrespective of the LLM used, a Mann-Whitney U test was performed. For multiple comparisons, Bonferroni correction was applied. The significance threshold was adjusted to p < 0.0063 for comparisons across all eight questions, p < 0.017 for comparisons across all three LLMs, and p < 0.05 was used for single comparisons. Fleiss kappa was used to assess interrater reliability.

Table 1 Questions Answered by ChatGPT, Bard, and LLaMA about MEK/BRAF Inhibitors.
Full size table

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Alowais SA, Alghamdi SS, Alsuhebany N, Alqahtani T, Alshaya AI, Almohareb SN, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ. 2023;23:689.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Clusmann J, Kolbinger FR, Muti HS, Carrero ZI, Eckardt J-N, Laleh NG, et al. The future landscape of large language models in medicine. Commun Med (Lond). 2023;3:141.

    Article  PubMed  Google Scholar 

  3. Li X, Hu X, Qi X, Yu L, Zhao W, Heng P-A, et al. Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis. IEEE Trans Med Imaging. 2021;40:2284–94.

    Article  PubMed  Google Scholar 

  4. Huang X, Wang H, She C, Feng J, Liu X, Hu X, et al. Artificial intelligence promotes the diagnosis and screening of diabetic retinopathy. Front Endocrinol (Lausanne). 2022;13:946915.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Dr. Berkenstock is supported by the Dracopolous and Hankins Uveitis Funds.

Author information

Authors and Affiliations

  1. The Drexel University College of Medicine, Philadelphia, PA, USA

    Nila Kirupaharan, Camellia Edalat & Rayna Marshall

  2. Department of Ophthalmology, Mayo Clinic, Rochester, MN, USA

    Lauren A. Dalvin

  3. Gavin Herbert Eye Institute, University of California, Irvine, CA, USA

    Kapil Mishra

  4. University of California San Diego, San Diego, CA, USA

    Hannah Xu

  5. Memorial Sloan Kettering Cancer Center, New York, NY, USA

    Jasmine H. Francis

  6. The Wilmer Eye Institute, Division of Ocular Immunology, the Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Meghan Berkenstock

Authors
  1. Nila Kirupaharan
    View author publications

    Search author on:PubMed Google Scholar

  2. Camellia Edalat
    View author publications

    Search author on:PubMed Google Scholar

  3. Lauren A. Dalvin
    View author publications

    Search author on:PubMed Google Scholar

  4. Kapil Mishra
    View author publications

    Search author on:PubMed Google Scholar

  5. Rayna Marshall
    View author publications

    Search author on:PubMed Google Scholar

  6. Hannah Xu
    View author publications

    Search author on:PubMed Google Scholar

  7. Jasmine H. Francis
    View author publications

    Search author on:PubMed Google Scholar

  8. Meghan Berkenstock
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Nila Kirupaharan: Conceptualisation, Data Curation, Investigation, Writing—Original Draft; Camellia Edalat: Conceptualisation, Data Curation, Writing—Original Draft; Lauren A. Dalvin: Conceptualisation, Supervision, Writing —reviewing and editing; Kapil Mishra: Conceptualisation, Supervision, Writing — reviewing and editing; Rayna Marshall: Conceptualisation, Data Curation, Writing—Original Draft; Hannah Xu: Conceptualisation, Data Curation—Original Draft; Jasmine H. Francis: Supervision, Writing—reviewing and editing; Meghan Berkenstock: Conceptualisation, Methodology, Supervision, Writing—reviewing and editing.

Corresponding author

Correspondence to Meghan Berkenstock.

Ethics declarations

Competing interests

Nila Kirupaharan: none, Camellia Edalat: none, Lauren A. Dalvin: Supported by CTSA Grant Number KL2 TR002379 from the National Center for Advancing Translational Science (NCATS), Consultant for Ideaya Biosciences, Kapil Mishra: none, Rayna Marshall: none, Hannah Xu: none, Jasmine H. Francis: none, Meghan Berkenstock: consultant for Eyepoint pharmaceuticals, Seagen Incorporated, Sanofi, Abbvie, and Glaxo Smith Kline Pharmaceuticals.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kirupaharan, N., Edalat, C., Dalvin, L.A. et al. Ocular toxicities associated with MEK/BRAF inhibitors: assessing the accuracy and completeness of large language models. Eye (2025). https://doi.org/10.1038/s41433-025-03961-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41433-025-03961-5

Search

Advanced search

Quick links