Abstract
The necessity for recommendation models that can capture both semantic information and device-mediated learner interactions has increased due to the rapid growth of IoT-aware e-learning environments. IoT-enhanced learning in this context refers to intelligent learning platforms that continuously create and log heterogeneous interaction data, including session dynamics, access patterns across linked devices, and engagement behaviors. This work introduces a coherent hybrid framework that combines an Actor–Critic reinforcement learning agent optimized using Proximal Policy Optimization (PPO) with BERT-based semantic encoding. By combining textual content with context-aware interaction logs gathered from intelligent learning platforms, the method creates richer learner representations. While a Mahalanobis distance module offers correlation-aware similarity cues to enhance resilience under sparse and high-dimensional data, these representations allow the Actor–Critic agent to constantly improve its recommendation policy. The usefulness of the suggested framework for IoT-aware intelligent e-learning systems is demonstrated by experiments conducted on three public MOOC datasets, which show steady improvements over robust baselines.
Data availability
The datasets utilized and/or examined in the current investigation are accessible from the corresponding author upon reasonable request.
References
Hoi, S. C., Sahoo, D., Lu, J. & Zhao, P. Online learning: A comprehensive survey. Neurocomputing 459, 249–289 (2021).
Madni, S. H. H. et al. Factors influencing the adoption of IoT for E-learning in higher educational institutes in developing countries. Front. Psychol. 13, 915596 (2022).
Kapoor, K. edX courses dataset 2021 [Dataset]. Kaggle. (2021). https://www.kaggle.com/datasets/khusheekapoor/edx-courses-dataset-2021
Tahir, S., Hafeez, Y., Abbas, M. A., Nawaz, A. & Hamid, B. Smart learning objects retrieval for E-Learning with contextual recommendation based on collaborative filtering. Educ. Inform. Technol. 27 (6), 8631–8668 (2022).
Torkashvand, A., Jameii, S. M. & Reza, A. Deep learning-based collaborative filtering recommender systems: a comprehensive and systematic review. Neural Comput. Appl. 35 (35), 24783–24827 (2023).
Huang, C. Q. et al. XKT: toward explainable knowledge tracing model with cognitive learning theories for questions of multiple knowledge concepts. IEEE Trans. Knowl. Data Eng. 36 (11), 7308–7325 (2024).
Embarak, O. An adaptive paradigm for smart education systems in smart cities using the internet of behaviour (IoB) and explainable artificial intelligence (XAI). In 2022 8th International Conference on Information Technology Trends (ITT) (pp. 74–79). IEEE. (2022).
Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H. & Alharbi, H. The COVID-19 pandemic and E-learning: challenges and opportunities from the perspective of students and instructors. J. Comput. High. Educ. 34 (1), 21–38 (2022).
Zeng, F., Tang, R. & Wang, Y. User personalized recommendation algorithm based on GRU network model in social networks. Mobile information systems, 2022(1), 1487586. (2022).
Huang, F., Bei, Y., Yang, Z., Jiang, J., Chen, H., Shen, Q., … Yu, P. S. (2025, March).Large language model simulator for cold-start recommendation. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining (pp. 261–270).
Nikolakopoulos, A. N., Ning, X., Desrosiers, C. & Karypis, G. Trust your neighbors: A comprehensive survey of neighborhood-based methods for recommender systems. Recommender Syst. Handb., 39–89. (2021).
Roy, D. & Dutta, M. A systematic review and research perspective on recommender systems. J. Big Data. 9 (1), 59 (2022).
Wang, Y., Ma, W., Zhang, M., Liu, Y. & Ma, S. A survey on the fairness of recommender systems. ACM Trans. Inform. Syst. 41 (3), 1–43 (2023).
Romero, C. (ed, S.) Educational data mining and learning analytics: An updated survey. Wiley interdisciplinary reviews: Data Min. Knowl. discovery 10 3 e1355 (2020).
Xu, H., Zhao, N., Xu, N., Niu, B. & Zhao, X. Reinforcement learning-based dynamic event-triggered prescribed performance control for nonlinear systems with input delay. Int. J. Syst. Sci., 1–16. (2025).
Perifanis, V. & Efraimidis, P. S. Federated neural collaborative filtering. Knowl. Based Syst. 242, 108441 (2022).
Thorat, S. A., Ashwini, G. & Seema, M. Survey on collaborative and content-based recommendation systems. In 2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 1541–1548). IEEE. (2023).
Bai, T., Wen, J. R., Zhang, J. & Zhao, W. X. A neural collaborative filtering model with interaction-based neighborhood. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 1979–1982). (2017).
Zhao, X. et al. Deep reinforcement learning for list-wise recommendations. arXiv preprint arXiv :180100209. (2017).
Alshareet, O. & Awasthi, A. Collaborative filtering in the age of AI: foundations, innovations, and emerging trends. Computing 107 (11), 1–56 (2025).
Biswas, P. K. & Liu, S. A hybrid recommender system for recommending smartphones to prospective customers. Expert Syst. Appl. 208, 118058 (2022).
Yuan, H. & Hernandez, A. A. User cold start problem in recommendation systems: A systematic review. IEEE access. 11, 136958–136977 (2023).
Lu, J., Wu, D., Mao, M., Wang, W. & Zhang, G. Recommender system application developments: a survey. Decis. Support Syst. 74, 12–32 (2015).
Xiao, J. et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv :170804617. (2017).
Guo, H., Tang, R., Ye, Y., Li, Z. & He, X. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv :170304247. (2023).
Chen, X., Yao, L., McAuley, J., Zhou, G. & Wang, X. Deep reinforcement learning in recommender systems: A survey and new perspectives. Knowl. Based Syst. 264, 110335 (2023).
Chen, M. et al. Off-policy actor-critic for recommender systems. In Proceedings of the 16th ACM Conference on Recommender Systems (pp. 338–349). (2022).
Wang, Y., Ma, D., Ma, J. & Jin, Q. HGCR: A heterogeneous graph-enhanced interactive course recommendation scheme for online learning. IEEE Trans. Learn. Technol. 17, 364–374 (2023).
Sun, F. et al. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management (pp. 1441–1450). (2019), November.
Deng, W., Zhu, P., Chen, H., Yuan, T. & Wu, J. Knowledge-aware sequence modelling with deep learning for online course recommendation. Inf. Process. Manag. 60 (4), 103377 (2023).
Chen, J. et al. SR-HetGNN: session-based recommendation with heterogeneous graph neural network. Knowl. Inf. Syst. 66 (2), 1111–1134 (2024).
Chinnadurai, J., Karthik, A., Ramesh, J. V. N., Banerjee, S., Rajlakshmi, P. V., Rao,K. V., … Rajaram, A. (2024). Enhancing online education recommendations through clustering-driven deep learning. Biomedical Signal Processing and Control, 97, 106669.
Chen, X., Wang, X., Wang, Y., Liu, D. & Zhang, W. Leveraging deep learning and graph analysis for enhanced course recommendations in online education. Sci. Rep. 15 (1), 18623 (2025).
Sattar, A. & Bacciu, D. Graph neural network for context-aware recommendation. Neural Process. Lett. 55 (5), 5357–5376 (2023).
Pulido-Gaytan, L. B., Tchernykh, A., Cortés-Mendoza, J. M., Babenko, M. & Radchenko, G. A survey on privacy-preserving machine learning with fully homomorphic encryption. In Latin American High Performance Computing Conference (pp. 115–129). Cham: Springer International Publishing. (2020).
THU-KEG. MOOCCube dataset [Dataset]. GitHub. (2020). https://github.com/THU-KEG/MOOCCubeX
NTHU MOOCs dataset. [Dataset]. GitHub. (2021). https://github.com/lingm5038-ctrl/NTHU-MOOCs-Data
Funding
The authors did not obtain any financial assistance for this study.
Author information
Authors and Affiliations
Contributions
All authors contributed to the conception and design of the study. Xia Chunqin and Wu Peixi conducted data collection, simulation, and analysis.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Ethical approval
Not relevant.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Chunqin, X., Peixi, W. A hybrid actor–critic and BERT framework for intelligent course recommendation in IoT-aware e-learning systems. Sci Rep (2026). https://doi.org/10.1038/s41598-026-40952-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-40952-2
