Abstract
Study design
Feasibility study.
Objective
The objective of this study is to explore the feasibility of capturing egocentric (first person) video recordings in the home of individuals with cervical spinal cord injury (SCI) for hand function evaluation.
Setting
Community-based study in Toronto, Ontario, Canada.
Methods
Three participants with SCI recorded activities of daily living (ADLs) at home without the presence of a researcher. Information regarding recording characteristics and compliance was obtained as well as structured and semi-structured interviews involving privacy, usefulness, and usability. A video processing algorithm capable of detecting interactions between the hand and objects was applied to the home recordings.
Results
In all, 98.58 ± 1.05% of the obtained footage was usable and included four to eight unique activities over a span of 3–7 days. The interaction detection algorithm yielded an F1 score of 0.75 ± 0.15.
Conclusions
Capturing ADLs using an egocentric camera in the home environment after SCI is feasible. Considerations regarding privacy, ease of use of the devices, and scheduling of recordings are provided.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
Data availability
Please contact the corresponding author for data availability.
References
Anderson KD. Targeting recovery: priorities of the spinal cord-injured population. J Neurotrauma. 2004;21:1371–83.
Linacre JM, Heinemann AW, Wright BD, Granger CV, Hamilton BB. The structure and stability of the functional independence measure. Arch Phys Med Rehabil. 1994;75:127–32.
Heinemann AW, Linacre JM, Wright BD, Hamilton BB, Granger C. Relationships between impairment and physical disability as measured by the functional independence measure. Arch Phys Med Rehabil. 1993;74:566–73.
Catz A, Itzkovich M, Agranov E, Ring H, Tamir A. SCIM Spinal Cord Independence Measure: a new disability scale for patients with spinal cord lesions. Spinal Cord. 1997;35:850.
Itzkovich M, Gelernter I, Biering-Sorensen F, Weeks C, Laramee MT, Craven BC, et al. The Spinal Cord Independence Measure (SCIM) version III: reliability and validity in a multi-center international study. Disabil Rehabil. 2007;29:1926–33. https://doi.org/10.1080/09638280601046302.
Bradburn NM, Rips LJ, Shevell SK. Answering autobiographical questions: the impact of memory and inference on surveys. Science. 1987;236:157–61.
Adams SA, Matthews CE, Ebbeling CB, Moore CG, Cunningham JE, Fulton J, et al. The effect of social desirability and social approval on self-reports of physical activity. Am J Epidemiol. 2005;161:389–98. https://doi.org/10.1093/aje/kwi054.
Prince SA, Adamo KB, Hamel ME, Hardt J, Gorber SC, Tremblay M. A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act. 2008;5:56.
Marino Ralph J. Domains of outcomes in spinal cord injury for clinical trials to improve neurological function. J Rehabil Res Dev. 2007;44:113.
Noorkõiv M, Rodgers H, Price CI. Accelerometer measurement of upper extremity movement after stroke: a systematic review of clinical studies. J Neuroeng Rehabil. 2014;11:144.
Lemmens RJM, Timmermans AAA, Janssen-Potten YJM, Pulles SA, Geers RP, Bakx WG, et al. Accelerometry measuring the outcome of robot-supported upper limb training in chronic stroke: a randomized controlled trial. PLoS ONE. 2014;9:e96414. https://doi.org/10.1371/journal.pone.0096414.
Waddell Kimberly J, Strube Michael J, Bailey RR, Klaesner JW, Birkenmeier RL, Dromerick AW, et al. Does task-specic training improve upper limb performance in daily life poststroke? Neurorehabil Neural Repair. 2017;31:290–300. https://doi.org/10.1177/1545968316680493.
Brogioli M, Schneider S, Popp WL, Albisser U, Brust AK, Velstra IM, et al. Monitoring upper limb recovery after cervical spinal cord injury: insights beyond assessment scores. Front Neurol. 2016;7:142. https://doi.org/10.3389/fneur.2016.00142.
Brogioli M, Popp WL, Albisser U, Brust AK, Frotzler A, Gassert R, et al. Novel sensor technology to assess independence and limb-use laterality in cervical spinal cord injury. J Neurotrauma. 2016;33:1950–7. https://doi.org/10.1089/neu.2015.4362.
Likitlersuang J, Zariffa J. Interaction detection in egocentric video: towards a novel outcome measure for upper extremity function. IEEE J Biomed Health Inform. 2018;22:561–9. https://doi.org/10.1109/JBHI.2016.2636748. Epub 7 Dec 2016.
Likitlersuang J, Sumitro ER, Cao T, Visée RJ, Kalsi-Ryan S, Zariffa J. Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home. J Neuroeng Rehabil. 2019;16:83.
Li C, Kitani KM. Pixel-level hand detection in ego-centric videos. In: IEEE conference on computer vision and pattern recognition (CVPR). Columbus, Ohio: IEEE; 2013. p. 3570–7.
Li C, Kitani KM. Model recommendation with virtual probes for egocentric hand detection. In: IEEE international conference computer vision (ICCV). Sydney, Australia: IEEE; 2013. p. 2624–31.
Serra G, Camurri M, Baraldi L, Benedetti M, Cucchiara R. Hand segmentation for gesture recognition in EGO-vision. In: Proceedings of the 3rd ACM international workshop on interactive multimedia on mobile & portable devices. Barcelona, Spain: ACM; 2013. p. 31–6.
Betancourt A, López MM, Regazzoni CS, Rauterberg M. A sequential classier for hand detection in the framework of egocentric vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. Washington, DC, United States: IEEE Computer Society; 2014. p. 586–91.
Fathi A, Ren X, Rehg JM. Learning to recognize objects in egocentric activities. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). Colorado Springs, Colorado: IEEE; 2011. p. 3281–8.
Fathi A, Rehg JM. Modeling actions through state changes. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR). Portland, Oregon: IEEE; 2013. p. 2579–86.
Pirsiavash H, Ramanan D. Detecting activities of daily living in first-person camera views. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). Providence, Rhode Island: IEEE; 2012. p. 2847–54.
Ren X, Philipose M. Egocentric recognition of handled objects: benchmark and analysis. In: IEEE computer society conference on Computer vision and pattern recognition workshops. Miami, Florida: IEEE; 2009. p. 1–8.
Zariffa J, Popovic MR. Hand contour detection in wearable camera video using an adaptive histogram region of interest. J Neuroeng Rehabil. 2013;10:114.
Khan AU, Borji A. Analysis of hand segmentation in the wild. arXiv:1803.03317 [Preprint]. 2018. https://arxiv.org/abs/1803.03317.
Bambach S, Lee S, Crandall DJ, Yu C. Lending a hand: detecting hands and recognizing activities in complex egocentric interactions. In: 2015 IEEE international conference on computer vision (ICCV). IEEE; 2015. p. 1949–57.
Kalsi-Ryan S, Beaton D, Curt A, et al. The graded redened assessment of strength sensibility and prehension: reliability and validity. J Neurotrauma. 2012;29:90–914.
Likitlersuang J, Sumitro ER, Theventhiran P, Kalsi-Ryan S, Zariffa J. Views of individuals with spinal cord injury on the use of wearable cameras to monitor upper limb function in the home and community. J Spinal Cord Med. 2017;40:706–14.
Redmon J, Farhadi A. YOLO9000: better, faster, stronger. arXiv [Preprint]. 2017. https://arxiv.org/abs/1612.08242.
Visée RJ, Likitlersuang J, Zariffa J. An effective and efficient method for detecting hands in egocentric videos for rehabilitation applications. arXiv:1908.10406 [Preprint]. 2019. https://arxiv.org/abs/1908.10406.
Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2016;39:1137–49.
Acknowledgements
The authors would like to thank Gregory Wong for his valuable assistance in the data labeling process. The authors would also like to thank all the participants of the study.
Funding
This study was supported in part by the Natural Sciences and Engineering Research Council of Canada (RGPIN-2014-05498), the Praxis Spinal Cord Institute (G2015-30), and the Ontario Early Researcher Award (ER16-12-013).
Author information
Authors and Affiliations
Contributions
JL was responsible for designing the protocol, recruiting the participants, collecting and analyzing data, interpreting results, creating figure/table of findings, and writing the paper. RJV contributed to the computer vision methods used in the analysis. SKR and JZ contributed to designing and supervising the study. All authors edited and approved the paper.
Corresponding author
Ethics declarations
Conflict of interest
SKR is a lead developer of GRASSP and owns the company that manufactures this product. The GRASSP is used as measure in this work, however, no licensing fee is applied for academic use.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Likitlersuang, J., Visée, R.J., Kalsi-Ryan, S. et al. Capturing hand use of individuals with spinal cord injury at home using egocentric video: a feasibility study. Spinal Cord Ser Cases 7, 17 (2021). https://doi.org/10.1038/s41394-021-00382-w
Received:
Revised:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41394-021-00382-w
