Abstract
Continual automated pain detection from facial expressions using computer vision during procedural sedation could optimize sedation dose titration and minimize pain. A prospective observational study was conducted. Participants’ faces were recorded during interventional radiology procedures performed with procedural sedation. Simultaneous pain assessments were made by a nurse using a sedation state assessment scale. Videos from 70 participants were used to train and evaluate a pain detection model using the Swin Transformer architecture. The model demonstrated an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.79 and an Area Under the Precision–Recall Curve (AP) of 0.53. The model reliably identified pain events for many participants, with sharp increases in predicted probabilities closely aligning with actual pain occurrences. Exploratory subgroup analyses suggested variability in performance across Fitzpatrick skin tone categories (highest in categories 3–4), but subgroup sizes were limited and these findings require validation in larger, independent cohorts. Automated pain detection systems for procedural sedation using this model would require a high threshold that minimizes false-positive alerts for pain to limit the risk of alarm fatigue. Alternatively, user interfaces that display predicted probabilities over time without alerts may be preferable until further refinements are made to enhance model performance and fairness across the diverse population of patients who receive procedural sedation.
Similar content being viewed by others
Data availability
Due to privacy issues related to the sensitive nature of the data used in this study where patients were undergoing medical procedures, the original images or data that could be used to reproduce the images is not available. The minimum dataset and code that could be used to reproduce the results of statistical analyses performed to evaluate the model can be requested from the corresponding author.
References
Glare, P., Aubrey, K. R. & Myles, P. S. Transition from acute to chronic pain after surgery. Lancet 393, 1537–1546 (2019).
Ashraf, A., Yang, A. & Taati, B. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) 1–5.
Gkikas, S. & Tsiknakis, M. Automatic assessment of pain based on deep learning methods: A systematic review. Comput. Methods Programs Biomed. 231, 107365 (2023).
Rezaei, S. et al. Unobtrusive pain monitoring in older adults with dementia using pairwise and contrastive training. IEEE J. Biomed. Health Inform. 25, 1450–1462. https://doi.org/10.1109/JBHI.2020.3045743 (2021).
Zarghami, Y., Mafeld, S., Conway, A. & Taati, B. Pain detection in masked faces during procedural sedation. In IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) (2023). https://doi.org/10.1109/FG57933.2023.10042502.
Hadjistavropoulos, T. & Craig, K. D. A theoretical framework for understanding self-report and observational measures of pain: A communications model. Behav. Res. Ther. 40, 551–570 (2002).
Herr, K. et al. Pain assessment in the nonverbal patient: Position statement with clinical practice recommendations. Pain Manag. Nurs. 7, 44–52 (2006).
Arbour, C. & Gélinas, C. Are vital signs valid indicators for the assessment of pain in postoperative cardiac surgery ICU adults?. Intensive Crit. Care Nurs. 26, 83–90 (2010).
Gélinas, C. & Arbour, C. Behavioral and physiologic indicators during a nociceptive procedure in conscious and unconscious mechanically ventilated adults: Similar or different?. J. Crit. Care 24, 628-e7 (2009).
Cravero, J. P. et al. Validation of the pediatric sedation state scale. Pediatrics 139, e20162897–e20162897 (2017).
Conway, A. et al. Automating sedation state assessments using natural language processing. J. Nurs. Scholarsh. 57, 17–27. https://doi.org/10.1111/jnu.12968 (2025).
Lugaresi, C. et al. Mediapipe: A framework for building perception pipelines. In Proceedings of 3rd Workshop on Computer Vision for AR/VR at CVPR (2019).
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D. & Zhai, X. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of International Conference on Learning Representations (ICLR) (2021).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (2016).
Xie, S., Girshick, R., Dollar, P., Tu, Z. & He, K. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 5987–5995 (2017).
Liu, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV) 10012–10022 (2021).
Yan, W.-J., Wu, Q., Liang, J., Chen, Y.-H. & Fu, X. How fast are the leaked facial expressions: The duration of micro-expressions. J. Nonverbal Behav. 37, 217–230 (2013).
Taati, B. et al. Algorithmic bias in clinical populations—evaluating and improving facial analysis technology in older adults with dementia. IEEE Access 7, 25527–25534 (2019).
Stopyn, R. J., Moturu, A., Taati, B. & Hadjistavropoulos, T. Real-time evaluation of an automated computer vision system to monitor pain behavior in older adults. J. Rehabil. Assist. Technol. Eng. 12, 20556683251313762. https://doi.org/10.1177/20556683251313762 (2025).
Yuan, X. et al. Occluded facial pain assessment in the ICU using action units guided network. IEEE J. Biomed. Health Inform. 28, 438–449 (2023).
Wang, L., Wang, Z., Xu, A. & Liu, S. In 2024 8th International Conference on Biomedical Engineering and Applications (ICBEA) 44–49 (IEEE).
Yuan, X. et al. In 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO) 723–728 (IEEE).
Conway, A., Rolley, J., Page, K. & Fulbrook, P. Issues and challenges associated with nurse-administered procedural sedation and analgesia in the cardiac catheterisation laboratory: A qualitative study. J. Clin. Nurs. 23, 374–384 (2014).
Cornelis, F. et al. Sedation and analgesia in interventional radiology: Where do we stand, where are we heading and why does it matter?. Diagn. Interv. Imaging. 100, 753–762 (2019).
Conway, A., Page, K., Rolley, J. X. & Worrall-Carter, L. Nurse-administered procedural sedation and analgesia in the cardiac catheter laboratory: An integrative review. Int. J. Nurs. Stud. 48, 1012–1023 (2011).
Conway, A., Chang, K., Mafeld, S. & Sutherland, J. Midazolam for sedation before procedures in adults and children: A systematic review update. Syst. Rev. 10, 1–12 (2021).
Subbaswamy, A., Schulam, P. & Saria, S. In The 22nd International Conference on Artificial Intelligence and Statistics 3118–3127 (PMLR).
Yang, Y. et al. Expert recommendation on collection, storage, annotation, and management of data related to medical artificial intelligence. Intell. Med. 3, 144–149 (2023).
Monares, M., Tang, Y., Raina, R. & de Sa, V. R. Analyzing biases in AU activation estimation toward fairer facial expression recognition. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2023).
Mafeld, S. et al. Avoiding and managing error in interventional radiology practice: Tips and tools. Can. Assoc. Radiol. J. 71, 528–535. https://doi.org/10.1177/0846537119899215 (2020).
Winters, B. D. et al. Technological distractions (Part 2): A summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit. Care Med. 46, 130–137 (2018).
Conway, A., Rolley, J. X., Page, K. & Fulbrook, P. Trends in nurse-administered procedural sedation and analgesia across Australian and New Zealand cardiac catheterisation laboratories: Results of an electronic survey. Aust. Crit. Care. 27, 4–10. https://doi.org/10.1016/j.aucc.2013.05.003 (2014).
Triantafyllou, K. et al. Sedation practices in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) survey. Endoscopy 56, 964–974 (2024).
Hall, D. et al. The landscape of pediatric procedural sedation in the United Kingdom and Irish emergency departments; an international survey study. Paediatr. Neonatal. Pain. 7, e12132 (2025).
Zupin, L. et al. Effectiveness of pharmacological procedural sedation in children with autism spectrum disorder: A systematic review and meta-analysis. Acta Paediatr. 113, 2363–2377 (2024).
Schneider, P., Lautenbacher, S. & Kunz, M. Sex differences in facial expressions of pain: Results from a combined sample. Pain (2022).
Acknowledgements
The authors would like to thank the AGE-WELL Network of Centres of Excellence, the Vector Institute for Artificial Intelligence, and the KITE Research Institute, Toronto Rehabilitation Institute – University Health Network. This study received funding from the Data Sciences Institute & Temerty Centre for Artificial Intelligence Research and Education in Medicine (University of Toronto).
Funding
This study received funding from the Data Sciences Institute & Temerty Centre for Artificial Intelligence Research and Education in Medicine (University of Toronto).
Author information
Authors and Affiliations
Contributions
A.C. and S.M. conceived the study, A.C., S.M., and B.T. designed the study, A.C., S.M., and B.T. organized funding, M.G.R. collected data, Y.Z. conducted the analyses and Y.Z. and A.C. wrote the first draft of the manuscript. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Zarghami, Y., Rad, M.G., Mafeld, S. et al. Computer vision for pain detection during procedural sedation. Sci Rep (2026). https://doi.org/10.1038/s41598-026-45130-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-45130-y
