Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Patient wisdom should be incorporated into health AI to avoid algorithmic paternalism

Nature Medicine volume 29, pages 765–766 (2023)Cite this article

Subjects

The first interactive computer, ELIZA, was created by Joseph Weizenbaum in 1972, who remarked: “Most of the damage that the computer could potentially cause depends less on what the computer actually can or cannot do than on the qualities that the public attributes to the computer.” As artificial intelligence (AI) advances into medicine, narratives of “AI versus clinicians,” and “AI outperforms doctors” dominate public discourse, sometimes positioning AI as a superior form of knowledge. As a result, some clinicians worry about the relevance of clinical judgment, patients get conflicting messages about whom to trust, and the general uncertainty about where AI fits within the evidence-based medicine paradigm continues to hinder translation into the clinic1.

The discourse around AI as superior has similarities to medical paternalism. Paternalism is captured by the idiom “the doctor knows best,” meaning that the physician, by virtue of their medical knowledge and training, is best situated to decide for another person what they should do. Paternalism can be recapitulated when AI is positioned as the highest form of evidence, replacing the all-knowing doctor with the all-knowing AI. Algorithmic paternalism is reflected in language—for example, efforts to improve trust in AI might imply that clinicians should learn to defer to AI. As another example, pitting of AI versus clinicians suggests that the two perform tasks that are directly comparable.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

References

  1. James, C. A., Wachter, R. M. & Wooliscroft, J. O. JAMA 327, 1333–1334 (2022).

    Article  PubMed  Google Scholar 

  2. Tschandl, P. et al. Nat. Med. 26, 1229–1234 (2020).

    Article  CAS  PubMed  Google Scholar 

  3. Gaube, S. et al. NPJ Digit. Med. 4, 31 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency 295–305 (Association for Computing Machinery, 2020).

  5. Jacobs, M. et al. Transl. Psychiatry 11, 108 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ghassemi et al. Lancet Digit. Health 3, e745–e750 (2021).

    Article  CAS  PubMed  Google Scholar 

  7. Birhane, A. Patterns (N.Y.) 2, 100205 (2021).

    Article  PubMed  Google Scholar 

  8. Collins, P. H. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment (Routledge, 2002).

  9. Baria, A. T. & Cross, K. Preprint at https://doi.org/10.48550/arXiv.2107.14042 (2021).

  10. Weizenbaum, J. Computer Power and Human Reason: From Judgment to Calculation (W. H. Freeman, 1976).

Download references

Author information

Authors and Affiliations

  1. Bioethics Department, The Hospital for Sick Children, Toronto, Ontario, Canada

    Melissa D. McCradden & Roxanne E. Kirsch

  2. Genetics & Genome Biology, Peter Gilgan Centre for Research and Learning, The Hospital for Sick Children, Toronto, Ontario, Canada

    Melissa D. McCradden

  3. Division of Clinical and Public Health, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    Melissa D. McCradden

  4. Critical Care Medicine Department, The Hospital for Sick Children, Toronto, Ontario, Canada

    Roxanne E. Kirsch

  5. Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada

    Roxanne E. Kirsch

Authors
  1. Melissa D. McCradden
    View author publications

    Search author on:PubMed Google Scholar

  2. Roxanne E. Kirsch
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Melissa D. McCradden.

Ethics declarations

Competing interests

M.D.M. and R.K. have no conflicts of interest related to this manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McCradden, M.D., Kirsch, R.E. Patient wisdom should be incorporated into health AI to avoid algorithmic paternalism. Nat Med 29, 765–766 (2023). https://doi.org/10.1038/s41591-023-02224-8

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41591-023-02224-8

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing