Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Reporting standards for the use of large language model-linked chatbots for health advice

Nature Medicine volume 29, page 2988 (2023)Cite this article

Subjects

Modern artificial intelligence (AI)-based chatbots such as ChatGPT leverage the technology of large language models (LLMs) and natural language processing1. These chatbots use natural language processing to enable user interactions via a chat-based interface that produces LLM-generated text. Their performance has improved as LLM-linked chatbots are increasingly being trained with online sources, and their impressive qualities have garnered interest in their application for healthcare purposes1.

Physicians and patients regularly use the internet for health advice2,3, and the wide availability of LLM-linked chatbots has raised concerns for patient safety due to the accuracy of their output4. Healthcare professionals have a fiduciary duty in supporting patients to access accurate information, while avoiding false or misleading sources. For this reason, there has been substantial research interest in investigating whether LLM-linked chatbots can provide accurate clinical advice. However, the reporting of the design, conduct and analysis of this growing body of research is inconsistent.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

References

  1. Thirunavukarasu, A. J. et al. Nat. Med. 29, 1930–1940 (2023).

    Article  CAS  PubMed  Google Scholar 

  2. Amante, D. J. et al. J. Med. Internet Res. 7, e106 (2015).

    Article  Google Scholar 

  3. Wang, C. et al. Med. Internet Res. 25, e48009 (2023).

    Article  Google Scholar 

  4. Mikalef, P. et al. Health Info. Libr. J. 34, 58–73 (2017).

    Article  PubMed  Google Scholar 

  5. Lee, T. C. et al. Gastroenterology. 165, 509–511 (2023).

    Article  PubMed  Google Scholar 

  6. Rahsepar, A. A. et al. Radiology 307, e230922 (2023).

    Article  PubMed  Google Scholar 

  7. Emilie, S. H. et al. Surgery 174, 1273–1275 (2023).

    Article  Google Scholar 

  8. Qu, R. W. et al. OTO Open. 7, e67 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Zhou, Z. et al. Eur Urol. 84, 355–356 (2023).

    Article  PubMed  Google Scholar 

  10. Moher, D. et al. PLoS Med. 7, e1000217 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of General Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada

    Bright Huo, Tyler McKechnie & Yung Lee

  2. USC Institute of Urology and Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA

    Giovanni E. Cacciamani

  3. AI Center at USC Urology, University of Southern California, Los Angeles, CA, USA

    Giovanni E. Cacciamani

  4. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK

    Gary S. Collins

  5. UK EQUATOR Centre, University of Oxford, Oxford, UK

    Gary S. Collins

  6. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada

    Tyler McKechnie & Gordon Guyatt

  7. Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA

    Yung Lee

  8. Department of Medicine, McMaster University, Hamilton, Ontario, Canada

    Gordon Guyatt

Authors
  1. Bright Huo
    View author publications

    Search author on:PubMed Google Scholar

  2. Giovanni E. Cacciamani
    View author publications

    Search author on:PubMed Google Scholar

  3. Gary S. Collins
    View author publications

    Search author on:PubMed Google Scholar

  4. Tyler McKechnie
    View author publications

    Search author on:PubMed Google Scholar

  5. Yung Lee
    View author publications

    Search author on:PubMed Google Scholar

  6. Gordon Guyatt
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Bright Huo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huo, B., Cacciamani, G.E., Collins, G.S. et al. Reporting standards for the use of large language model-linked chatbots for health advice. Nat Med 29, 2988 (2023). https://doi.org/10.1038/s41591-023-02656-2

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41591-023-02656-2

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research