Tsui et al. [1]. reported their experience with using a popular online artificial intelligence (AI) chatbot, ChatGPT for ophthalmic symptoms triage. The authors scripted ten prompts for common ophthalmic scenarios that patients may experience including retinal detachment, exudative age-related macular degeneration, ocular surface disease, etc. The authors found that ChatGPT only provided suitable and precise responses for eight out of the ten responses with the two remaining answers being both unsuitable and imprecise [1]. Since the release of this study, OpenAI released GPT-4 which is reported to have a broader knowledge base, image analysis capabilities, as well as greater problem solving abilities than the previous version [2].

We examined the performance of GPT-4 by re-testing its ability to respond to the two scenarios that it struggled with previously. We then graded GPT-4’s response to these prompts, and the responses produced were now both precise and suitable (Table 1). For example, in response to the prompt “I have macular degeneration and things look wavy today”, GPT-4 now recommends seeking immediate medical attention if experiencing severe or sudden visual changes, which is a more precise and suitable answer clinically. GPT-4 also re-states a second time that it strongly recommends the patient to seek medical attention. Previously, ChatGPT would state the importance of talking to a doctor, but then mention the importance of wearing sunglasses and quitting smoking which is an unsuitable and imprecise response to a potentially irreversible loss of vision.

Table 1 Appropriateness of responses by GPT-4.
Full size table

Log in or create a free account to read this content

Gain free access to this article, as well as selected content from this journal and more on nature.com

Access through your institution

or

Sign in or create an account
Continue with Google
Continue with ORCiD