Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Chatbots’ performance in premature ejaculation questions: a comparative analysis of reliability, readability, and understandability

International Journal of Impotence Research (2025)Cite this article

Subjects

Abstract

This study aimed to evaluate the reliability, readability, and understandability of chatbot responses to frequently asked questions about premature ejaculation, and to assess the contributions, potential risks, and limitations of artificial intelligence. Fifteen questions were selected using data from Google Trends and posed to the chatbots Copilot, Gemini, ChatGPT4o, ChatGPT4oPlus, and DeepSeek-R1. Reliability was evaluated using the Global Quality Scale(GQS) by two experts, readability was assessed with the Flesch Kincaid Reading Ease(FKRE), Flesch Kincaid Grade Level(FKGL), Gunning Fog Index(GFI), and Simple Measure of Gobbledygook(SMOG), and understandability was evaluated using the Patient Educational Materials Assessment Tool for Printable Materials(PEMAT-P). Additionally, the consistency of source citations was examined. The GQS were as follows: Copilot: 3.96 ± 0.66, Gemini: 3.66 ± 0.78, ChatGPT4o: 4.83 ± 0.23, ChatGPT4oPlus: 4.83 ± 0.29, DeepSeek-R1:4.86 ± 0.22 (p < 0.001). The PEMAT-P were as follows: Copilot: 0.70 ± 0.05, Gemini: 0.72 ± 0.04, ChatGPT4o: 0.83 ± 0.03, ChatGPT4oPlus: 0.77 ± 0.06, DeepSeek-R1:0.79 ± 0.06 (p < 0.001). While ChatGPT4oPlus and DeepSeek-R1 scored higher for reliability and understandability, all chatbots performed at an acceptable level (≥70%). However, readability scores were above the recommended level for the target audience. Instances of low reliability or unverified sources were noted, with no significant differences between the chatbots. Chatbots provide highly reliable and informative responses regarding premature ejaculation; however, it is evident that there are significant limitations that require improvement, particularly concerning readability and the reliability of sources.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Data availability

All data generated or analysed during this study are included in this published article [and its supplementary information files]. We confirm that any accession codes or datasets mentioned in the article will be made publicly available shortly after proof submission and that database records will be updated with publication details once available.

References

  1. Porst H, Montorsi F, Rosen RC, Gaynor L, Grupe S, Alexander J. The premature ejaculation prevalence and attitudes (PEPA) survey: prevalence, comorbidities, and professional help-seeking. Eur Urol. 2007;51:816–23. https://doi.org/10.1016/j.eururo.2006.07.004

    Article  PubMed  Google Scholar 

  2. Saitz TR, Serefoglu EC. The epidemiology of premature ejaculation. Transl Androl Urol. 2016;5:409–15. https://doi.org/10.21037/tau.2016.05.11

    Article  PubMed  PubMed Central  Google Scholar 

  3. Athanasiadis L. Premature ejaculation: is it a biogenic or a psychogenic disorder? Sex Marital Ther. 1998;13:241–55. https://doi.org/10.1080/02674659808406567

    Article  Google Scholar 

  4. Althof S. Prevalence, characteristics and implications of premature ejaculation/rapid ejaculation. J Urol. 2006;175:842–8. https://doi.org/10.1016/S0022-5347(05)00341-1

    Article  PubMed  Google Scholar 

  5. Martin C, Nolen H, Podolnick J, Wang R. Current and emerging therapies in premature ejaculation: where we are coming from, where we are going. Int J Urol. 2017;24:40–50. https://doi.org/10.1111/iju.13202

    Article  PubMed  Google Scholar 

  6. Shechter A, Gruenwald I. New technologies developed for treatment of premature ejaculation. Int J Impot Res. 2024 Nov;36:700–5. https://doi.org/10.1038/s41443-024-00875-w

    Article  PubMed  PubMed Central  Google Scholar 

  7. Hall AK, Bernhardt JM, Dodd V, Vollrath MW. The digital health divide: evaluating online health information access and use among older adults. Health Educ Behav. 2015;42:202–9. https://doi.org/10.1177/1090198114547815

    Article  PubMed  Google Scholar 

  8. Papaharitou S, Athanasiadis L, Nakopoulou E, Kirana P, Portseli A, Iraklidou M, et al. Erectile dysfunction and premature ejaculation are the most frequently self-reported sexual concerns: profiles of 9536 men calling a helpline. Eur Urol. 2006;49:557–63. https://doi.org/10.1016/j.eururo.2005.09.023

    Article  PubMed  Google Scholar 

  9. Al Sliman R, Kuru TH, Struck JP, Heinrichs AL, Arndt K, Salem J, et al. Usage of the internet and digital health among urological patients: findings from a Cross-Sectional study. Urol Int. 2025:1-7. https://doi.org/10.1159/000544873

  10. Şahin MF, Keleş A, Özcan R, Doğan Ç, Topkaç EC, Akgül M, et al. Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation. Sex Med. 2024;12:qfae036. https://doi.org/10.1093/sexmed/qfae036

    Article  PubMed  PubMed Central  Google Scholar 

  11. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol. 2007;102:2070–7. https://doi.org/10.1111/j.1572-0241.2007.01325.x

    Article  PubMed  Google Scholar 

  12. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. University of Central Florida; 1975, Report No.: Institute for Simulation and Training; 56. https://doi.org/10.21236/ADA006655

  13. Gunning R The Technique of Clear Writing. New York: McGraw-Hill; 1952.

  14. Mc Laughlin GH. SMOG grading-a new readability formula. J Read. 1969;12:639–46.

    Google Scholar 

  15. Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns. 2014;96:395–403. https://doi.org/10.1016/j.pec.2014.05.027

    Article  PubMed  PubMed Central  Google Scholar 

  16. Wang H, Fu T, Du Y, Gao W, Huang K, Liu Z, et al. Scientific discovery in the age of artificial intelligence. Nature. 2023;620:47–60. https://doi.org/10.1038/s41586-023-06221-2

    Article  CAS  PubMed  Google Scholar 

  17. Haug CJ, Drazen JM. Artificial intelligence and machine learning in clinical medicine, 2023. N Engl J Med. 2023;388:1201–8. https://doi.org/10.1056/NEJMra2302038

    Article  CAS  PubMed  Google Scholar 

  18. Baturu M, Solakhan M, Kazaz TG, Bayrak O. Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship. Int J Impot Res. 2025;37:310–4. https://doi.org/10.1038/s41443-024-00898-3

    Article  PubMed  Google Scholar 

  19. Nashed JY, Liblik K, Dergham A, Witherspoon L, Flannigan R. Artificial intelligence in andrology: a new frontier in male infertility diagnosis and treatment. Curr Urol Rep. 2025;26:29. https://doi.org/10.1007/s11934-025-01257-5

    Article  PubMed  Google Scholar 

  20. Xiong Y, Zhang Y, Zhang F, Wu C, Qin F, Yuan J. Applications of artificial intelligence in the diagnosis and prediction of erectile dysfunction: a narrative review. Int J Impot Res. 2023;35:95–102. https://doi.org/10.1038/s41443-022-00528-w

    Article  PubMed  Google Scholar 

  21. Warren CJ, Edmonds VS, Payne NG, Voletti S, Wu SY, Colquitt J, et al. Prompt matters: evaluation of large language model chatbot responses related to Peyronie’s disease. Sex Med. 2024;12:qfae055. https://doi.org/10.1093/sexmed/qfae055

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Gaziosmanpasa Training and Research Hospital, Department of Urology, Istanbul, Turkey

    S. Gonultas, S. Kardas, M. Gelmis, A. H. Kinik, M. Ozalevli, M. G. Kose & B. Arslan

  2. Derindere Hospital, Department of Urology, Istanbul, Turkey

    S. Sulejman

  3. Bagcilar Training and Research Hospital, Department of Urology, Istanbul, Turkey

    S. Yentur

Authors
  1. S. Gonultas
    View author publications

    Search author on:PubMed Google Scholar

  2. S. Kardas
    View author publications

    Search author on:PubMed Google Scholar

  3. M. Gelmis
    View author publications

    Search author on:PubMed Google Scholar

  4. A. H. Kinik
    View author publications

    Search author on:PubMed Google Scholar

  5. M. Ozalevli
    View author publications

    Search author on:PubMed Google Scholar

  6. M. G. Kose
    View author publications

    Search author on:PubMed Google Scholar

  7. S. Sulejman
    View author publications

    Search author on:PubMed Google Scholar

  8. S. Yentur
    View author publications

    Search author on:PubMed Google Scholar

  9. B. Arslan
    View author publications

    Search author on:PubMed Google Scholar

Contributions

SG, SK, MG, AHK, MO, MGK, SS, SY, BA: Designing, drafting, revising, providing study conception, data acquisition, analysis, and interpretation. All authors approved the final version of the manuscript to be published.

Corresponding author

Correspondence to S. Gonultas.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical approval

As the artificial intelligence chatbots utilized in this study are publicly accessible, and the study did not involve human or animal participants, ethical approval was not required.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gonultas, S., Kardas, S., Gelmis, M. et al. Chatbots’ performance in premature ejaculation questions: a comparative analysis of reliability, readability, and understandability. Int J Impot Res (2025). https://doi.org/10.1038/s41443-025-01179-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41443-025-01179-3

Search

Advanced search

Quick links