Abstract
Chronic stress is an important threat in Public Health, as it negatively impacts both the Body and Mind. Current methods for measuring and identifying stress rely largely on individuals providing subjective assessments or measuring isolated physiological parameters, thereby limiting the accuracy and consistency of these approaches. This study proposes a novel approach to objectively measuring an individual’s level of stress, by combining deep transfer learning methods for detecting psychological stress measured using Electroencephalogram (EEG) and Electrocardiogram (ECG) data. More specifically, this new method uses three pre-trained neural network backbones—VGG16, EfficientNetB0, and ResNeXt50—utilized together, to create a unified system capable of merging information from multiple streams of data in real-time. EEG data is converted to time-frequency maps using wavelet transformations and ECG data uses time-series variability (i.e. patterns of how the heart beats) combined with raw, unfiltered data. An advanced fusion layer uses attention weights to intelligently combine these two data sources, allowing for improved accuracy of stress assessment. Using the WESAD and CASE datasets, both of which were collected from 35 subjects while they were in a neutral (control), tense, and positive state, our method performs at 95.7% accuracy in identifying between these three conditions, which is significantly greater than the accuracy rates of either the EEG-only (82.3%) or ECG-only (85.6%) methods or individual networks. Furthermore, this system is highly flexible and has demonstrated the capability to successfully operate across numerous testing conditions, while additionally demonstrating that EEG signals enhance ECG stress assessment and vice versa. Therefore, this new approach provides a highly reliable way to support medical diagnosis, employee wellness programs, and personalized psychological support.
Similar content being viewed by others
Data availability
Both datasets used in this study are publicly available research datasets:1. WESAD (Wearable Stress and Affect Detection Dataset)The dataset is publicly available for academic research and can be accessed at: https://uni-siegen.sciebo.de/s/HGdUkoNlW1Ub0Gx2. CASE (Continuously Annotated Signals of Emotion Dataset)The dataset is publicly available for research purposes and can be accessed at: https://doi.org/10.6084/m9.figshare.8869157 https://springernature.figshare.com/articles/dataset/CASE_Dataset-full/8869157 Additional processed data generated during the study are available from the corresponding author upon reasonable request.
References
Yedukondalu, J. et al. Deep fusion of vgg19 and convmixer with superlet transform for cognitive load detection. IEEE Trans. Consum. Electron. (2025).
Myrick, C., Ghosh, I., Jayarajah, K. & Roy, N. CALM: Multimodal cognitive load assessment framework via engineered and explainable features. In 2025 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) 213–218 (IEEE Computer Society, 2025).
Li, Y., Li, K., Zhang, J., Wang, S. & Wang, D. CLNet: A lightweight real-time network for monitoring pilots’ cognitive load based on multi-scale spatiotemporal convolution. Int. J. Hum. Comput. Interact. 1–22 (2025).
Kholiya, R. & Ahirwal, M. K. An end-to-end brain–computer interface for mental workload classification based on EEGNet model and data augmentation. In International Conference On Artificial Intelligence, Computing, IOT and Data Analytics 37–47 (Springer, 2026).
Ma, Y., Li, Z., Chen, L., Liu, Q. & Yang, L. Workload recognition of real navigation: A deep learning approach integrating EEG scalogram and eye-tracking data. Ocean Eng. 348, 124106 (2026).
Thanaraj, S., Balodi, A., Anand, R. S. & Rawat, A. Automatic boundary detection and severity assessment of mitral regurgitation. Biomed. Signal. Process. Control. 82, 104616 (2023).
Balodi, A., Anand, R. S., Dewal, M. L. & Rawat, A. Computer-aided classification of the mitral regurgitation using multiresolution local binary pattern. Neural Comput. Appl. 32(7), 2205–2215 (2020).
Balodi, A., Anand, R. S., Dewal, M. L. & Rawat, A. Severity analysis of mitral regurgitation using discrete wavelet transform. IETE J. Res. 69(1), 209–219 (2023).
Wang, A., Yang, H., Wang, J., Yang, H. & He, D. Driver cognitive load estimation in conditional driving with aligned attention-enabled multimodal fusion. Transp. Res. Part. C: Emerg. Technol. 183, 105471 (2026).
Li, Y., Li, K., Wang, S., Wu, H. & Li, P. A spatiotemporal separable graph convolutional network for oddball paradigm classification under different cognitive-load scenarios. Expert Syst. Appl. 262, 125303 (2025).
Li, Y., Li, K., Wang, S., Wu, H. & Li, P. A spatiotemporal separable graph convolutional network for oddball paradigm classification under different cognitive-load scenarios (2025).
Li, Y., Li, K., Wang, S., Wu, H. & Li, P. A Spatiotemporal Separable Graph Convolutional Network for Oddball Paradigm Classification Under Different Cognitive-Load Scenarios. Available at SSRN 4725369.
Pradeep, M., Sasi, A., Farrukh, N., Venugopal, R. & Sherly, E. Cross-modal computational model of brain-heart interactions via HRV and EEG feature. arXiv preprint arXiv:2601.06792 (2026).
Tan, S. et al. Automatic detection and prediction of epileptic EEG signals based on nonlinear dynamics and deep learning: A review. Front. NeuroSci. 19, 1630664 (2025).
Cheng, S. & Dong, J. IMCATN: Transformer Based Learning With Multi-Scale Information Fusion for Driver Cognitive Load Recognition. Int. J. Hum. Comput. Interact. 1–19 (2025).
Babu, A. R., Bhargav, S., Reddy, S. L. & Maick, B. M. A novel approach for classification of EEG subjects using hybrid machine learning algorithm. In 2025 Fourth International Conference on Power, Control and Computing Technologies (ICPC2T) 472–477 (IEEE, 2025).
AlArnaout, Z., Zaki, C., Kotb, Y., AlAkkoumi, M. & Mostafa, N. Exploiting heart rate variability for driver drowsiness detection using wearable sensors and machine learning. Sci. Rep. 15(1), 24898 (2025).
Ding, Y. et al. Neural Decoding for EEG-BCI: From conventional machine learning to deep learning models. Brain Hemorrhages (2026).
Nouri, Z., Charmin, A., Kalbkhani, H. & Barghandan, S. Multivariate synchrosqueezing transform and time-frequency attention for mental workload classification from EEG signals. Sci. Rep. (2026).
Wang, J., Wang, Y., Nie, W. & Yuan, Q. Explainable end-to-end seizure prediction via dynamic multiscale cross-band fusion filter network. Int. J. Neural Syst. (2025).
Raj, V. A., Parupudi, T., Thalengala, A. & Nayak, S. G. A comprehensive review of deep learning models for denoising EEG signals: challenges, advances, and future directions. Discover Appl. Sci. 7(11), 1268 (2025).
Zhou, T., Shu, L., Zhang, Z. & Han, J. Tyee: A unified, modular, and fully-integrated configurable toolkit for intelligent physiological health care. In Proceedings of the 33rd ACM International Conference on Multimedia 13628–13631 (2025).
Elgendi, M. et al. ECG sonification methods for robust and generalizable clinical decision support. npj Digit. Med. (2025).
Basnet, N. & Zahabi, M. Real-time cognitive workload assessment using non-intrusive methods: A systematic review. Ergonomics 1–26 (2025).
Singhal, P. & Yadav, R. K. Improved clustering techniques for paediatric cerebral palsy gait assessment during rehabilitation. Int J. Inf. Tecnol (2024).
Elgendy, M. A., Eletriby, S., Keshk, A. & Sakr, M. Unravelling Schizophrenia: An Attention-Based Deep Learning Approach for Detection Using EEG Signals. IJCI Int. J. Computers Inform. 12(1), 67–84 (2025).
Singhal, P. & Yadav, R. K. GAIT: A computing survey towards approaches, methodologies and applications. 2023 International Conference on IoT, Communication and Automation Technology (ICICAT) 1–7 (2023).
Funding
Open access funding provided by Manipal Academy of Higher Education, Manipal
Author information
Authors and Affiliations
Contributions
Rakesh Kumar, Sivanesan Bala Krishnan, and Rakesh Kumar Yadav conceptualized the problem statement and developed the proposed solution. Dilip Kumar Jang Bahadur Saini, Prasun Chakrabarti, and Arun Balodi structured the manuscript and contributed to drafting and technical refinement. Shwetha V critically reviewed the manuscript and provided valuable inputs for improving the overall quality and clarity of the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kumar, R., Krishnan, S.B., Yadav, R.K. et al. An attention-based multimodal deep learning framework integrating EEG and ECG for enhanced stress detection. Sci Rep (2026). https://doi.org/10.1038/s41598-026-44499-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-44499-0
