Abstract
Fatigue driving is one of the primary causes of traffic accidents, posing a serious threat not only to the driver’s life and property but also to the safety of surrounding vehicles and pedestrians. Therefore, accurate and efficient detection of driver fatigue is essential for preventing traffic incidents. This study proposes a novel driver fatigue detection framework that integrates AlphaPose* with a Long Short-Term Memory (LSTM) network. To enhance the performance of AlphaPose, the original human detector is replaced with an optimized YOLOv11n model, which incorporates a Hybrid Pooling Fusion Block (HPFB) to improve feature representation and meet the requirements of keypoint estimation. Furthermore, a multi-view data processing strategy based on the Driving Monitoring Dataset (DMD) is introduced to capture driver behavior from frontal, lateral, and hand views. To effectively model the temporal dynamics of driver behavior, a fatigue behavior monitoring network is designed using LSTM. Comparative experiments demonstrate that the proposed AlphaPose*-LSTM-based system achieves superior performance in fatigue detection tasks compared to existing state-of-the-art approaches.
Data availability
Data are contained within the article.The experimental data employed in this study are derived from the open-source DMD video dataset39.
References
Global status report on road safety 2018. Geneva: World Health Organization. License: CC BY- NC-SA 3.0 IGO,2018.
AAA Foundation for Traffic Safety. 2019 traffic safety culture index. AAA Foundation for Traffic Safety, Technical report, (2020).
Alameen, S. A. & Alhothali, A. M. A lightweight driver drowsiness detection system using 3DCNN With LSTM. Comput. Syst. Sci. Eng. 44 (1), 895–912 (2023).
National Highway Traffic Safety Administration. Traffic safety facts 2015. (2015). https://crashstats.nhtsa.dot.gov/Api/Public/Publication/812384.
Vanlaar Ward, H. et al. Fatigued and drowsy driving: A survey of attitudes, opinions and behaviors. J. Saf. Res. 39, 303–309. https://doi.org/10.1016/j.jsr.2007.12.007 (2008).
Chen, C. S., Lu, J. & Ma, K. K. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics): Preface. Lect Notes Comput. Sci. 10116, V–VI (2017).
Gaëtan Merlhiot, M. Bueno How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review. Accid. Anal. Prev. 170, 106536. (2022). https://doi.org/10.1016/j.aap.2021.106536
Lian, Z. et al. Driving fatigue detection based on hybrid electroencephalography and eye tracking. IEEE J. Biomed. Health Inf. 28 (11), 6568–6580. (2024). https://doi.org/10.1109/JBHI.2024.3446952
Ardabili, S. Z. et al. A novel approach for automatic detection of driver fatigue using EEG signals based on graph convolutional networks. Sensors 24, 364. https://doi.org/10.3390/s24020364 (2024).
Siddhad, G. et al. Awake at the wheel: Enhancing automotive safety through EEG-based fatigue detection. In: International Conference on Pattern Recognition. Springer, Cham, pp. 340–353. (2025).
Zhou, X. et al. An EEG channel selection framework for driver drowsiness detection via interpretability guidance. In: 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, pp 1–5. (2023).
Rezaee, Q. et al. Driver drowsiness detection with commercial EEG headsets. In: 2022 10th RSI International Conference on Robotics and Mechatronics (ICRoM). IEEE, pp 546–550. (2022).
Halomoan, J., Ramli, K., Sudiana, D., Gunawan, T. S. & Salman, M. ECG-based driving fatigue detection using heart rate variability analysis with mutual information. Information 14, 539. https://doi.org/10.3390/info14100539 (2023).
Zeng, C. et al. Driver fatigue detection using heart rate variability features from 2-minute electrocardiogram signals while accounting for sex differences. Sensors 24, 4316. https://doi.org/10.3390/s24134316 0/s24134316 (2024).
Xiong, H. et al. Detection of driver drowsiness level using a hybrid learning model based on ECG signals. Biomed. Tech. (Berl). 69 (2), 151–165. (2023). . https://doi.org/10.1515/bmt-2023-0193
Wang, L., Song, F., Zhou, T. H., Hao, J. & Ryu, K. H. EEG and ECG-based multi-sensor fusion computing for real-time fatigue driving recognition based on feedback mechanism. Sensors 23, 8386. https://doi.org/10.3390/s23208386 (2023).
Satti, A. T., Kim, J., Yi, E., Cho, H. Y. & Cho, S. Microneedle array electrode-based wearable EMG system for detection of driver drowsiness through steering wheel grip. Sens. (Basel). 21 (15), 5091. https://doi.org/10.3390/s21155091 (2021).
Mahmoodi, M. & Nahvi, A. Investigation of sleep deprivation effect on driver’s electromyography signal features in a driving simulator. J. Sleep. Sci. 3 (3–4), 53–62 (2019).
Karthikeyan, V. et al. A narrative vehicle protection representation for vehicle speed regulator under driver exhaustion–a study. (2014). arXiv preprint arXiv:1402.3657.
Li, Z., Chen, L., Peng, J. & Wu, Y. Automatic detection of driver fatigue using driving operation information for transportation safety. Sensors 17, 1212. https://doi.org/10.3390/s17061212 (2017).
Li, Z., Li, S. E., Li, R., Cheng, B. & Shi, J. Online detection of driver fatigue using steering wheel angles for real driving conditions. Sensors 17, 495. https://doi.org/10.3390/s17030495 (2017).
Zhendong Lan, J. et al. Driving fatigue detection based on fusion of EEG and vehicle motion information. Biomed. Signal Process. Control 92, 106031. https://doi.org/10.1016/j.bspc.2024.106031 (2024)
Shaik, Md. E. A systematic review on detection and prediction of driver drowsiness. Transp. Res. Interdisciplinary Perspect. 21, 100864. (2023). https://doi.org/10.1016/j.trip.2023.100864
Zhao, S., Peng, Y., Wang, Y. & Li, G. Mohammed Al-Mahbashi1 Lightweight YOLOM-Net for automatic identification and real-time detection of fatigue driving. Computers Mater. Continua. 82 (3), 4995–5017. https://doi.org/10.32604/cmc.2025.059972 (2025).
Li, A. et al. Driver fatigue detection and human-machine cooperative decision-making for road scenarios. Multimed Tools Appl. 83, 12487–12518. https://doi.org/10.1007/s11042-023-15994-7 (2024).
Akin, A. & Kalkan, H. Detecting driver fatigue with eye blink behavior. arXiv preprint arXiv:2407.02222, (2024).
Dreißig, M. et al. Driver drowsiness classification based on eye blink and head movement features using the k-NN algorithm. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, : 889–896. (2020).
Shugang Liu, Y. et al. A driver fatigue detection algorithm based on dynamic tracking of small facial targets using YOLOv7. IEICE. Trans. Inf. Syst. E106-D (11), 1881–1890 (2023).
Nguyen, P. V., Smith, A. & Lee, B. Driver yawning detection system using key facial landmarks and mouth aspect ratio. In: Advances in Computer Vision. Lecture Notes in Computer Vision. Springer, (2022).
Dkhil, M. B., Wali, A. & Alimi, A. M. Towards a new system for drowsiness detection based on eye blinking and head posture estimation. (2018). arXiv preprint arXiv:1806.00360.
Lyu, J., Yuan, Z. & Chen, D. Long-term multi-granularity deep framework for driver drowsiness detection. (2018). arXiv preprint arXiv:1801.02325.
Albadawi, Y., AlRedhaei, A. & Takruri, M. Real-time machine learning-based driver drowsiness detection using visual features. J. Imaging. 9, 91. https://doi.org/10.3390/jimaging9050091 (2023).
Delwar, T. S. et al. AI- and deep learning-powered driver drowsiness detection method using facial analysis. Appl. Sci. 15, 1102. https://doi.org/10.3390/app15031102 (2025).
Yashaswini, N. L. et al. Journey tracker: driver alerting system with a deep learning approach. Front. Robot AI. 11, 1433795. https://doi.org/10.3389/frobt.2024.1433795 (2024).
Zhou, C. et al. Research on driver facial fatigue detection based on Yolov8 model. In: 2024 5th International Conference on Information Science, Parallel and Distributed Systems (ISPDS). IEEE, pp. 282–285. (2024).
Fang, H. S. et al. AlphaPose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE Trans. Pattern Anal. Mach. Intell. 45 (6), 7157–7173. (2023). https://doi.org/10.1109/TPAMI.2022.3222784
Hochreiter, S., & Schmidhuber, J. Long short-term memory. Neural Comput. 9 (8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 (1997).
Khanam, R. & Hussain, M. Yolov11: An overview of the key architectural enhancements. (2024). arXiv preprint arXiv:2410.17725.
Ortega, J. D. et al. Dmd: A large-scale multi-modal driver monitoring dataset for attention and alertness analysis. In: European Conference on Computer Vision. Cham: Springer International Publishing, : 387–405. (2020).
Maycock, G. Sleepiness and driving: the experience of U.K. car drivers. Accid Anal Prev. ;29(4):453 – 62. (1997). https://doi.org/10.1016/s0001-4575(97)00024-9. PMID: 9248503.
Palazzi, A. et al. Predicting the driver’s focus of attention: the DR (eye) VE project. IEEE Trans. Pattern Anal. Mach. Intell. 41 (7), 1720–1733 (2018).
Fang, J. et al. Driver attention prediction in driving accident scenarios. IEEE Trans. Intell. Transp. Syst. 23 (6), 4959–4971 (2021).
Xia, Y. et al. Predicting driver attention in critical situations. In: Asian conference on computer vision. Cham: Springer International Publishing, : 658–674. (2018).
Vijayan, V., & Pushpalatha, K. P. A comparative analysis of RootSIFT and SIFT methods for drowsy features extraction. Procedia Comput. Sci. 171, 436–445. (2020). https://doi.org/10.1016/j.procs.2020.04.046
Bobbie Seppelt, S., Seaman, L., Angell, B., Mehler & Reimer, B. Differentiating cognitive load using a modified version of AttenD. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘17). Association for Computing Machinery, New York, NY, USA, 114–122. (2017). https://doi.org/10.1145/3122986.3123019
Zhang, B. T., Chang, W. W. & Li, X. L. Fatigue driving detection based on spatial-temporal electroencephalogram features and parallel, vol. 23, pp. 315–325, (2023). https://doi.org/10.16097/j.cnki.1009-6744.2023.02.033
Schwarz, A., Haurilet, M., Martinez, M. & Stiefelhagen, R. DriveAHead—A large-scale driver head pose dataset. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HA, USA, 21–26; Volume 2017, pp. 1165–1174. (2017).
Roth, M. & Gavrila, D. M. DD-pose—A large-scale driver head pose benchmark. In Proceedings of the IEEE Intelligent Vehicles Symposium, Paris, France, 9–12 June 2019; Volume 2019, pp. 927–934.
Liu, D. et al. Toward extremely lightweight distracted driver recognition with distillation-based neural architecture search and knowledge transfer. IEEE Trans. Intell. Transp. Syst. 24 (1), 764–777 (2022).
Ohn-Bar, E. & Trivedi, M. M. The power is in your hands: 3D analysis of hand gestures in naturalistic video. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA, 2013, pp. 912–917, (2013). https://doi.org/10.1109/CVPRW.2013.134
Das, N., Ohn-Bar, E. & Trivedi, M. M. Onallenges, Data performance evaluation of driver hand detection algorithms: chset, and metrics. In: 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Gran Canaria, Spain, pp. 2953–2958, (2015). https://doi.org/10.1109/ITSC.2015.473
Ultralytics. Ultralytics YOLOv8. GitHub repository. (2023). https://github.com/ultralytics/ultralytics
Wang, A. et al. Yolov10: Real-time end-to-end object detection. Adv. Neural. Inf. Process. Syst. 37, 107984–108011 (2024).
Zhang, W. & Su, J. Driver yawning detection based on long short term memory networks. In Proceedings of IEEE symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA, pp 1–5 (2017).
Fei, Y., Li, B. & Wang, H. Long short-term memory network based fatigue detection with sequential mouth feature. In: Proceedings of International Symosium on Autonomous Systems (ISAS), Guangzhou, China, pp. 17–22, (2020).
Lee, C. & An, J. LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEG. Expert Syst. Appl. 213, 119032 (2023) https://doi.org/10.1016/j.eswa.2022.119032
Moredo, M. J. R., Celino, J. D. S. & Ibarra, J. B. G. Multi-feature long short-term memory facial recognition for real-time automated drowsiness observation of automobile drivers with raspberry Pi 4. Eng. Proc. 92, 52. https://doi.org/10.3390/engproc2025092052 (2025).
Xiao, W., Liu, H., Ma, Z., Chen, W. & Hou, J. F. P. I. R. S. T. Fatigue driving recognition method based on feature parameter images and a residual swin transformer. Sensors 24, 636. https://doi.org/10.3390/s24020636 (2024).
Krisna, G. S., Supriya, K. & Vardhan, J. Vision transformers and YoloV5 based driver drowsiness detection framework. (2022). arXiv preprint arXiv:2209.01401.
Acknowledgements
We are especially grateful to our team members for their excellent cooperation and patient support during the literature analysis process.
Funding
This research received no external funding.
Author information
Authors and Affiliations
Contributions
Conceptualization, Y.H. and D.W.; methodology, Y.H.; software, X.S.; validation, Y.H., X.S. and H.L.; formal analysis, Y.H.; investigation, Y.H.; resources, X.S.; data curation, X.S.; writing—original draft preparation, Y.H.; writing—review and editing, H.L.; visualization, X.S.; supervision, Y.H.; project administration, H.L.; funding acquisition, D.W. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Ethical approval
The experimental data employed in this study are derived from the open-source DMD video dataset39. Due to the limitations of this dataset, the proposed model is not directly applicable to nighttime or low-illumination environments. In practical applications, the parameters for evaluating fatigue should be adjusted based on specific data characteristics to ensure compliance with the requirements of the model. The primary application of this model lies in driver fatigue warning tasks. In real-world deployment, repeated testing of the model’s outcomes is necessary to ensure its robustness and reliability. This study provides reference technical indicators; however, actual applications should carefully consider the influence of environmental conditions and various external factors.
Conflict of interest
The author declares no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Hao, Y., Sun, X., Liu, H. et al. A fatigue driving detection method based on driver posture and facial state analysis. Sci Rep (2026). https://doi.org/10.1038/s41598-026-44994-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-44994-4
