Abstract
Concerns that AI tools may erode diagnostic reasoning contrast with claims that AI can foster higher-order thinking. This longitudinal study followed 372 medical students across 12 months of supervised rotations using an AI-assisted diagnosis system. AI-assisted diagnosis participation, AI literacy and medical critical thinking were assessed at baseline, 6 months and 12 months. Cross-lagged panel models examined prospective associations, statistical mediation by AI literacy and moderation by prior technological experience and learning goal orientation. Higher participation was associated with increases in AI literacy and critical thinking, and AI literacy statistically mediated the participation-to-critical thinking association. Indirect effects were stronger among students with greater technological experience and mastery-oriented goals and weaker among performance-oriented peers. Findings indicate that, within supervised clinical training, engagement with AI systems is associated with critical thinking development partly through enhanced AI literacy, supporting AI tools as educational resources under faculty guidance.
Similar content being viewed by others
Data availability
The datasets generated and analysed during the current study are not publicly available due to institutional data protection regulations, but de-identified data may be made available from the corresponding author on reasonable request and with approval from the relevant institutional bodies.
Code availability
All custom code used in this study is available from the corresponding author upon reasonable request. The longitudinal structural equation models were estimated in Mplus (Muthén & Muthén) using robust maximum likelihood estimation with full information maximum likelihood (FIML) for handling missing data. Descriptive and supplementary analyses were conducted in R (R Foundation for Statistical Computing). The Mplus syntax files and R scripts specify all variables, model parameters, and options used to generate and analyse the current datasets.
References
Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44–56 (2019).
Civaner, M. M., Uncu, Y., Bulut, F., Chalil, E. G. & Tatli, A. Artificial intelligence in medical education: a cross-sectional needs assessment. BMC Med. Educ. 22, 772 (2022).
Thesen, T. & Park, S. H. A generative AI teaching assistant for personalized learning in medical education. npj Digital Med. 8, 627 (2025).
Sutton, R. T. et al. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digital Med. 3, 17 (2020).
McCoy, L. G. et al. What do medical students actually need to know about artificial intelligence?. NPJ Digital Med. 3, 86 (2020).
Gaube, S. et al. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ Digital Med. 4, 31 (2021).
Gerlich, M. AI tools in society: impacts on cognitive offloading and the future of critical thinking. Societies 15, 6 (2025).
Cabral, B. P., Braga, L. A. M., Conte Filho, C. G. & Mota, F. Future use of AI in diagnostic medicine: 2-wave cross-sectional survey study. J. Med. Internet Res. 27, e53892 (2025).
Gonsalves, C. Generative AI’s impact on critical thinking: revisiting Bloom’s taxonomy. J. Mark. Educ. 46, 150–165 (2024).
Chan, K. S. & Zary, N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med. Educ. 5, e13930 (2019).
Long, D. & Magerko, B. What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. (ACM, 2020).
Karaca, O., Caliskan, S. A. & Demir, K. Medical artificial intelligence readiness scale for medical students (MAIRS-MS) – development, validity and reliability study. BMC Med. Educ. 21, 112 (2021).
Facione, P. A. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction (American Philosophical Association, 1990).
Fogel, A. L. & Kvedar, J. C. Artificial intelligence powers digital medicine. NPJ Digital Med. 1, 5 (2018).
Masters, K. Ethical use of artificial intelligence in health professions education: AMEE Guide No. 158. Med. Teach. 45, 574–584 (2023).
Sapci, A. H. & Sapci, H. A. Artificial intelligence education and tools for medical and health informatics students: systematic review. JMIR Med. Educ. 6, e19285 (2020).
Dweck, C. S. Motivational processes affecting learning. Am. Psychol. 41, 1040–1048 (1986).
Beik, A. & Cho, Y. Effects of goal orientation on online learning: a meta-analysis of differences in Korea and US. Curr. Psychol. 43, 1496–1506 (2024).
Pallant, J. L., Blijlevens, J., Campbell, A. & Jopp, R. Mastering knowledge: the impact of generative AI on student learning outcomes. Stud. Higher Educ. 5, 1–22 (2025).
Wang, S., Shi, G., Lu, M., Lin, R. & Yang, J. Determinants of active online learning in the smart learning environment: an empirical study with PLS-SEM. Sustainability 13, 9923 (2021).
Yang, X., Deng, S., Hu, W. & Deng, Y. Associations between social media use and academic adjustment among university students: an integrated variable-centered and person-centered analytical framework. Probl. Educ. 21st Century 83, 383–402 (2025).
Xin, Y., Shusheng, D., Weina, H. & Yan, D. How does digital connection shape cultural adaptation? The impact of social media use on cross-cultural adaptation of international students in China. J. Int. Stud. 15, 1–26 (2025).
Kolachalama, V. B. & Garg, P. S. Machine learning and medical education. NPJ Digital Med. 1, 54 (2018).
Fiedler, K., Schott, M. & Meiser, T. What mediation analysis can (not) do. J. Exp. Soc. Psychol. 47, 1231–1236 (2011).
Fiedler, K., Harris, C. & Schott, M. Unwarranted inferences from statistical mediation tests–an analysis of articles published in 2015. J. Exp. Soc. Psychol. 75, 95–102 (2018).
Cheung, M. W. L. & Lau, R. S. Testing mediation and suppression effects of latent variables: bootstrapping with structural equation models. Organ. Res. Methods 11, 296–325 (2008).
Preacher, K. J. & Hayes, A. F. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behav. Res. Methods 40, 879–891 (2008).
Klein, A. & Moosbrugger, H. Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika 65, 457–474 (2000).
Maslowsky, J., Jager, J. & Hemken, D. Estimating and interpreting latent variable interactions: a tutorial for applying the latent moderated structural equations method. Int. J. Behav. Dev. 39, 87–96 (2015).
Preacher, K. J., Curran, P. J. & Bauer, D. J. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. J. Educ. Behav. Stat. 31, 437–448 (2006).
Sweller, J., van Merriënboer, J. J. G. & Paas, F. Cognitive architecture and instructional design: 20 years later. Educ. Psychol. Rev. 31, 261–292 (2019).
Wang, W. Medical education in China: progress in the past 70 years and a vision for the future. BMC Med. Educ. 21, 453 (2021).
Zhang, S., Ganapathy Prasad, P. & Schroeder, N. L. Learning about AI: a systematic review of reviews on AI literacy. J. Educ. Comput. Res. 63, 07356331251342081 (2025).
Wu, W., Carroll, I. A. & Chen, P. Y. A single-level random-effects cross-lagged panel model for longitudinal mediation analysis. Behav. Res. Methods 50, 2111–2124 (2018).
Enders, C. K. Applied Missing Data Analysis (Guilford Publications, 2022).
Velasquez, G. & Zhang, Q. Cross-lagged panel mediation models with latent constructs: specification and estimation. Multivar. Behav. Res. 55, 142–143 (2020).
Zyphur, M. J. et al. From data to causes I: building a general cross-lagged panel model (GCLM). Organ. Res. Methods 23, 651–687 (2020).
MacCallum, R. C., Zhang, S., Preacher, K. J. & Rucker, D. D. On the practice of dichotomization of quantitative variables. Psychol. Methods 7, 19 (2002).
Acknowledgements
This work was supported by the 2021 Guangxi Philosophy and Social Science Planning Research Project (No. 21FKS028), the Health and Emergency Skills Training Center of Guangxi (No. HESTCG202304), and the 2022 Guangxi Higher Education Undergraduate Teaching Reform Project (No. 2022JGZ155). The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the manuscript. We are grateful to all participating students and clinical teaching staff for their time and support in completing the surveys. We also thank the administrative teams at the participating institutions for their assistance with coordinating data collection.
Author information
Authors and Affiliations
Contributions
Conceptualization: Y.X., D.Y., L.L.H. Methodology: Y.X., D.Y., L.L.H. Formal analysis: Y.X., D.Y., L.L.H. Investigation: Y.X., D.Y., L.M.Y., L.SR., L.L.H. Data curation: Y.X., D.Y., L.L.H. Writing – original draft: Y.X., D.Y., L.L.H. Writing – review & editing: Y.X., D.Y., L.LH., L.M.Y., L.S.R. Funding acquisition: Y.X., D.Y., L.L.H. Resources: L.L.H., L.M.Y., L.S.R. Supervision: D.Y., L.L.H. Project administration: D.Y., L.L.H.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Xin, Y., Yan, D., Shuren, L. et al. AI literacy mediates AI assisted diagnosis participation and critical thinking among medical students under supervision. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-02521-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41746-026-02521-9
