Abstract
Online education offers flexibility but often suffers from reduced learner engagement. This study developed an automated method to detect emotional engagement using an optimized Vision Transformer model with Transfer Learning. Facial data from 40 undergraduates produced a dataset of 71,185 labeled images across three engagement levels. The proposed model achieved 93.8% classification accuracy, surpassing conventional machine learning and deep learning baselines. Analysis showed engagement typically declined after six minutes of learning, with a modest rebound near session end. Pearson correlation revealed a significant positive relationship between engagement and learning outcomes, indicating that emotionally engaged learners achieved higher academic performance. These results demonstrate the feasibility of deep learning–based approaches for scalable monitoring of learner engagement and highlight the important role of emotional states in shaping online learning effectiveness. The findings provide practical insights for designing adaptive interventions to sustain attention and optimize digital learning environments.
Similar content being viewed by others
Data availability
The datasets generated and analyzed during the current study are not publicly available due to ethical restrictions related to identifiable facial data, but are available from the corresponding author upon reasonable request and subject to approval by the institutional ethics committee.
References
Venton, B. J. & Pompano, R. R. Strategies for enhancing remote student engagement through active learning. Anal. Bioanal. Chem. 413 (6), 1507–1512 (2021).
Pabba, C. & Kumar, P. An intelligent system for monitoring students’ engagement in large classroom teaching through facial expression recognition. Expert Syst., 39(1), (2022).
Sümer, Ö. et al. Multimodal Engagement Analysis from Facial Videos in the classroom (IEEE Transactions on Affective Computing, 2021).
Baker, R. S. et al. Better to be frustrated than bored: the incidence, persistence, and impact of learners’ cognitive–affective States during interactions with three different computer-based learning environments. Int. J. Hum. Comput. Stud. 68 (4), 223–241 (2010).
Bian, C. et al. Spontaneous facial expression database for academic emotion inference in online learning. IET Comput. Vision. 13 (3), 329–337 (2019).
Pekrun, R. et al. Achievement emotions and academic performance: longitudinal models of reciprocal effects. Child Dev. 88 (5), 1653–1670 (2017).
Whitehill, J. et al. The faces of engagement: automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect. Comput. 5 (1), 86–98 (2014).
Fredricks, J. A., Blumenfeld, P. C. & Paris, A. H. School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74 (1), 59–109 (2004).
Bosch, N., Chen, Y. & D’Mello, S. It’s written on your face: detecting affective states from facial expressions while learning computer programming[C]. Intelligent Tutoring Systems: 12th International Conference, ITS (2014).
D’Mello, S. K., Craig, S. D. & Graesser, A. C. Multimethod assessment of affective experience and expression during deep learning. Int. J. Learn. Technol. 4 (3–4), 165–187 (2009).
Means, B. & Toyama Yuki and Murphy, Robert and Bakia, Marianne and Jones, Karla. Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Project Report. Centre for Learning Technology[M]. Assoc. Learn. Technol., (2009).
Kaur, A. et al. Prediction and localization of student engagement in the wild[C]. 2018 digital image computing: techniques and applications (DICTA). IEEE, 1–8. (2018).
Kamath, A., Biswas, A. & Balasubramanian, V. A crowdsourced approach to student engagement recognition in e-learning environments[C]. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1–9. (2016).
Mukhopadhyay, M. et al. Facial emotion detection to assess Learner’s State of mind in an online learning system[C]. Proceedings of the 5th international conference on intelligent information technology. (2020).
Gupta, A. et al. Daisee: towards user engagement recognition in the wild. (2016).
Alyuz, N. et al. Towards an emotional engagement model: Can affective states of a learner be automatically detected in a 1:1 learning scenario?[C]. 24th ACM Conference on User Modeling, Adaptation and Personalization (UMAP). (2016).
D’Mello, S. K. & Graesser, A. Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model. User-Adapt. Interact. 20, 147–187 (2010).
Skinner, E. A. & Belmont, M. J. Motivation in the classroom: reciprocal effects of teacher behavior and student engagement across the school year. J. Educ. Psychol. 85 (4), 571–581 (1993).
Schmieder, A. A glossary of educational reform. J. Teacher Educ. 24 (1), 55–62 (1973).
Fisher, C. W. et al. Teaching behaviors, academic learning time, and student achievement: an overview. J. Classr. Interact. 17 (1), 2–15 (1981).
Connell, J. P., Wellborn, J. G. & Competence Autonomy, and relatedness: A motivational analysis of self-system processes. J. Personal. Soc. Psychol., (65):43–77. (1991).
Kahu, E. R. & Nelson, K. Student Engagement in the Educational Interface: Understanding the Mechanisms of Student success Vol. 37, 58–71 (Higher education research & development, 2018).
Aslan, S. et al. Human expert labeling process (HELP): towards a reliable higher-order user state labeling process and tool to assess student engagement. Educ. Technol., : 53–59. (2017).
Kuh, G. D. The National survey of student engagement: conceptual and empirical foundations. New. Dir. Institutional Res. 2009 (141), 5–20 (2010).
Whitehill, J. et al. The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans. Affect. Comput., (1):86–98. (2014).
Funding
National Natural Science Foundation of China (No. 62177032), “Research on the Autonomous Training and Evaluation Model for Pre-service Teachers’ Classroom Teaching Expression Competence.”
Author information
Authors and Affiliations
Contributions
Guanyu Chen is responsible for article writing and revision, as well as communication work.Guangxin Han is responsible for data calculation and processing, as well as icon creation and modification.Juan Niu has provided academic research and theoretical support for this study.Juhou He is the administrator of this project and provides guidance throughout the research process.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Chen, G., Han, G., Niu, J. et al. Understanding the impact of emotional engagement on learning outcomes in online education: an automated analysis approach. Sci Rep (2026). https://doi.org/10.1038/s41598-025-34871-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-025-34871-x
