Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Development and evaluation of a computer vision algorithm for quantification of children’s microactivities

Journal of Exposure Science & Environmental Epidemiology (2025)Cite this article

Abstract

Background

Estimates of microactivity (e.g., hand- and object-to-mouth contact) frequencies are essential for modeling children’s environmental exposures but are challenging to obtain due to the time and human costs of manually labeling behaviors from pre-recorded videos.

Objectives

We aim to develop and evaluate a computer vision model to quantify microactivities for young children.

Methods

The vision model was trained and validated using video footage (collected via four concurrent Go-Pro cameras) of 25 children 6–18 months playing in their homes in Baltimore, MD. We leveraged computer vision techniques to develop an algorithm to assess children’s pose by identifying and tracking 3D key points (e.g., locations of children’s eyes, hands, wrists, elbows, etc.). We enabled automatic measurement to track the distance between the child’s hands and mouth in every video frame. When the distance reached a minimum threshold, the model logged a “contact event.” We compared the timing and number of events for three microactivities (left- and right-hand-to-mouth, and object-to-mouth) yielded by the vision model to the outputs from comparable human behavioral coding.

Results

Our method recognizes children’s microactivities. The timing and number of contact events detected were accurate (96–99%) on a second-level basis with minimal counting errors (<0.04–2.18 per video). We observed higher rates of object-to-mouth contacts (mean = 27 contacts/h) compared to hand-to-mouth contacts (mean = 3 contacts/h).

Impact

  • This study developed and evaluated a computer vision method for accurately identifying and quantifying young children’s hand-to-mouth and object-to-mouth contacts from collected video, greatly reducing the costs and burden of generating microactivity data needed for soil and dust exposure modeling.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2: Flowchart illustrating the computer vision pipeline.

Similar content being viewed by others

Data availability

The dataset and codebook will be available from the authors upon request.

References

  1. Zartarian V, Xue J, Tornero-Velez R, Brown J. Children’s lead exposure: a multimedia modeling analysis to guide public health decision-making. Environ Health Perspect. 2017;125:097009.

    Article  PubMed  PubMed Central  Google Scholar 

  2. US EPA. Exposure Factors Handbook Chapter 5 2017 [Available from: https://www.epa.gov/expobox/exposure-factors-handbook-chapter-5.

  3. Panagopoulos Abrahamsson D, Sobus JR, Ulrich EM, Isaacs K, Moschet C, Young TM, et al. A quest to identify suitable organic tracers for estimating children’s dust ingestion rates. J Expo Sci Environ Epidemiol. 2021;31:70–81.

    Article  PubMed  CAS  Google Scholar 

  4. Ferguson A, Adelabu F, Solo-Gabriele H, Obeng-Gyasi E, Fayad-Martinez C, Gidley M, et al. Methodologies for the collection of parameters to estimate dust/soil ingestion for young children. Front Public Health. 2024;12.

  5. Xue J, Zartarian V, Moya J, Freeman N, Beamer P, Black K, et al. A meta-analysis of children’s hand-to-mouth frequency data for estimating nondietary ingestion exposure. Risk Anal. 2007;27:411–20.

    Article  PubMed  Google Scholar 

  6. Tsou M-C, Özkaynak H, Beamer P, Dang W, Hsi H-C, Jiang C-B, et al. Mouthing activity data for children aged 7 to 35 months in Taiwan. J Expo Sci Environ Epidemiol. 2015;25:388–98.

    Article  PubMed  Google Scholar 

  7. Freeman NCG, Jimenez M, Reed KJ, Gurunathan S, Edwards RD, Roy A, et al. Quantitative analysis of children’s microactivity patterns: the Minnesota Children’s Pesticide Exposure Study. J Expo Sci Environ Epidemiol. 2001;11:501–9.

    Article  CAS  Google Scholar 

  8. Tulve NS, Suggs JC, McCurdy T, Cohen Hubal EA, Moya J. Frequency of mouthing behavior in young children. J Expo Sci Environ Epidemiol. 2002;12:259–64.

    Article  Google Scholar 

  9. Rochat P (ed.) Object manipulation and exploration in 2-to 5-month-old infants 2001.

  10. Ruff HA. Infants’ manipulative exploration of objects: Effects of age and object characteristics. Dev Psychol. 1984;20:9–20.

    Article  Google Scholar 

  11. Palmer CF. The discriminating nature of infants’ exploratory actions. Dev Psychol. 1989;25:885–93.

    Article  Google Scholar 

  12. Malachowski LG, Needham AW. Infants exploring objects: a cascades perspective. Adv Child Dev Behav. 2023;64:39–68.

    Article  PubMed  Google Scholar 

  13. Whyte VA, McDonald PV, Baillargeon R, Newell KM. Mouthing and grasping of objects by young infants. Ecol Psychol. 1994;6:205–18.

    Article  Google Scholar 

  14. Moya J, Phillips L. A review of soil and dust ingestion studies for children. J Expo Sci Environ Epidemiol. 2014;24:545–54.

    Article  PubMed  Google Scholar 

  15. Beamer PI, Canales RA, Bradman A, Leckie JO. Farmworker children’s residential non-dietary exposure estimates from micro-level activity time series. Environ Int. 2009;35:1202–9.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  16. Beamer P, Key ME, Ferguson AC, Canales RA, Auyeung W, Leckie JO. Quantified activity pattern data from 6 to 27-month-old farmworker children for use in exposure assessment. Environ Res. 2008;108:239–46.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Black K, Shalat SL, Freeman NCG, Jimenez M, Donnelly KC, Calvin JA. Children’s mouthing and food-handling behavior in an agricultural community on the US/Mexico border. J Expo Sci Environ Epidemiol. 2005;15:244–51.

    Article  CAS  Google Scholar 

  18. Tsou M-C, Özkaynak H, Beamer P, Dang W, Hsi H-C, Jiang C-B, et al. Mouthing activity data for children age 3 to <6 years old and fraction of hand area mouthed for children age <6 years old in Taiwan. J Expo Sci Environ Epidemiol. 2018;28:182–92.

    Article  PubMed  Google Scholar 

  19. Kwong LH, Ercumen A, Pickering AJ, Unicomb L, Davis J, Luby SP. Age-related changes to environmental exposure: variation in the frequency that young children place hands and objects in their mouths. J Expo Sci Environ Epidemiol. 2020;30:205–16.

    Article  PubMed  Google Scholar 

  20. Ferguson AC, Canales RA, Beamer P, Auyeung W, Key M, Munninghoff A, et al. Video methods in the quantification of children’s exposures. J Expo Sci Environ Epidemiol. 2006;16:287–98.

    Article  PubMed  Google Scholar 

  21. Juberg DR, Alfano K, Coughlin RJ, Thompson KM. An observational study of object mouthing behavior by young children. Pediatrics. 2001;107:135–42.

    Article  PubMed  CAS  Google Scholar 

  22. Zartarian VG, Ferguson AC, Ong CG, Leckie JO. Quantifying videotaped activity patterns: video translation software and training methodologies. J Expo Anal Environ Epidemiol. 1997;7:535–42.

    PubMed  CAS  Google Scholar 

  23. Zartarian VG, Streicker J, Rivera A, Cornejo CS, Molina S, Valadez OF, et al. A pilot study to collect micro-activity data of two- to four-year-old farm labor children in Salinas Valley, California. J Expo Anal Environ Epidemiol. 1995;5:21–34.

    PubMed  CAS  Google Scholar 

  24. Ferguson A, Dwivedi A, Adelabu F, Ehindero E, Lamssali M, Obeng-Gyasi E, et al. Quantified activity patterns for young children in beach environments relevant for exposure to contaminants. Int J Environ Res Public Health. 2021;18.

  25. Oh HS, Ryu M. Hand-to-face contact of preschoolers during indoor activities in childcare facilities in the Republic of Korea. Int J Environ Res Public Health. 2022;19:13282.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Fang H-S, Li J, Tang H, Xu C, Zhu H, Xiu Y, et al. Alphapose: whole-body regional multi-person pose estimation and tracking in real-time. IEEE Trans Pattern Anal Mach Intell. 2022;45:7157–73.

    Article  Google Scholar 

  27. Wang J, Sun K, Cheng T, Jiang B, Deng C, Zhao Y, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell. 2020;43:3349–64.

    Article  Google Scholar 

  28. Loper M, Mahmood N, Romero J, Pons-Moll G, Black MJ. SMPL: a skinned multi-person linear model. Semin Graph Pap: Push Bound. 2023;2:851–66. p.

    Google Scholar 

  29. Huang X, Fu N, Liu S, Ostadabbas S (editors) Invariant representation learning for infant pose estimation with small data. In: Proceedings 16th international conference on automatic face and gesture recognition (FG 2021); IEEE; 2021.

  30. Cai Z, Yin W, Zeng A, Wei C, Sun Q, Yanjun W, et al. Smpler-X: scaling up expressive human pose and shape estimation. Adv Neural Inf Process Syst. 2024;36.

  31. Goel S, Pavlakos G, Rajasegaran J, Kanazawa A, Malik J. Humans in 4D: reconstructing and tracking humans with transformers. In: Proceedings IEEE/CVF international conference on computer vision (ICCV). 2023. pp 14783–94.

  32. BuildClinical. 2025 [https://www.buildclinical.com/].

  33. Szeliski R. Computer vision: algorithms and applications, 2nd ed. Switzerland: Springer; 2022.

  34. Joo HL, Liu H, Tan L, Gui L, Nabbe B, Matthews I, et al. Panoptic studio: a massively multiview system for social motion capture. In: Proceedings IEEE international conference on computer vision. 2015:3334–42.

  35. Dong JFQ, Jiang W, Yang Y, Huang Q, Bao H, Zhou X. Fast and robust multi-person 3 d pose estimation and tracking from multiple views. IEEE Trans Pattern Anal Mach Intell. 2021;44:6981–92.

    Article  Google Scholar 

  36. Ren S, He K, Girshick R, Sun J. Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;9199:2969239–50.

    Google Scholar 

  37. Contributors M. Openmmlab pose estimation toolbox and benchmark. 2020.

  38. Sun K, Xiao B, Liu D, Wang J, editors. Deep high-resolution representation learning for human pose estimation. In: Proceedings IEEE/CVF conference on computer vision and pattern recognition; 2019.

  39. Jin S, Xu L, Xu J, Wang C., Liu W, Qian C, et al. Whole-body human pose estimation in the wild. In: Proceedings 16th European conference on computer vision–ECCV 2020; 23–28 August; Glasgow, UK: Springer International Publishing; 2020. pp 196–214.

  40. Ren T, Liu S, Zeng A, Lin J, Li K, Cao H, et al. Grounded Sam: assembling open-world models for diverse visual tasks. Preprint at https://doi.org/10.48550/arXiv.2401.14159.

  41. Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, et al., editors. Segment anything. In: Proceedings IEEE/CVF international conference on computer vision; 2023.

  42. Easymocap—make human motion capture easier Github2021 https://github.com/zju3dv/EasyMocap.

  43. He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016.

  44. Bakeman R, Quera V. Behavioral observation. APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics. APA handbooks in psychology®. Washington, DC, US: American Psychological Association; 2012. p. 207–25.

  45. Bakeman R. Behavioral observation and coding. Handbook of research methods in social and personality psychology. New York, NY, US: Cambridge University Press; 2000. p. 138–59.

  46. Beamer PI, Luik CE, Canales RA, Leckie JO. Quantified outdoor micro-activity data for children aged 7–12-years old. J Expo Sci Environ Epidemiol. 2012;22:82–92.

    Article  PubMed  Google Scholar 

  47. Lopez-Galvez N, Claude J, Wong P, Bradman A, Hyland C, Castorina R, et al. Quantification and analysis of micro-level activities data from children aged 1-12 years old for use in the assessments of exposure to recycled tire on turf and playgrounds. Int J Env Res Public Health. 2022;19:2483.

  48. Groot EM, Lekkerkerk MC, Steenbekkers, LPA. Mouthing behaviour of young children; an observational study (summary report). RIVM report 613320 002. RIVM: Bilthoven, The Netherlands; 1998.

  49. Davis S MP, Kohler E, Wiggins C. Soil ingestion in children with PICA: Final Report (US EPA Cooperative Agreement CR 816334-01). Seattle, WA: Fred Hutchison Cancer Research Center; 1995.

  50. Hubal EAC, Sheldon LS, Burke JM, McCurdy TR, Berry MR, Rigas ML, et al. Children’s exposure assessment: a review of factors influencing Children’s exposure, and the data available to characterize and assess that exposure. Environ Health Perspect. 2000;108:475–86.

    Article  Google Scholar 

  51. Pacheco C, Mavroudi E, Kokkoni E, Tanner HG, Vidal R, editors. A detection-based approach to multiview action classification in infants. In: Proceedings 25th international conference on pattern recognition (ICPR); 2021.

  52. Chorney JM, McMurtry CM, Chambers CT, Bakeman R. Developing and modifying behavioral coding schemes in pediatric psychology: a practical guide. J Pediatr Psychol. 2014;40:154–64.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Dechemi A, Bhakri V, Sahin I, Modi A, Mestas J, Peiris P, et al., editors. BabyNet: a lightweight network for infant reaching action recognition in unconstrained environments to support future pediatric rehabilitation applications. In: Proceedings 30th IEEE international conference on robot & human interactive communication (RO-MAN); 8–12 August 2021.

  54. Manne SKR, Zhu S, Ostadabbas S, Wan M, editors. Automatic infant respiration estimation from video: a deep flow-based algorithm and a novel public benchmark. In: Proceedings international workshop on preterm, perinatal and paediatric image analysis. Springer; 2023.

  55. Zhu S, Wan M, Hatamimajoumerd E, Jain K, Zlota S, Kamath CV, et al., editors. A video-based end-to-end pipeline for non-nutritive sucking action recognition and segmentation in young infants. In: Proceedings medical image computing and computer assisted intervention—MICCAI 2023; Cham: Springer Nature Switzerland; 2023.

  56. Hesse N, Pujades S, Romero J, Black MJ, Bodensteiner C, Arens M, et al., editors. Learning an infant body model from RGB-D data for accurate full-body motion analysis. In: Proceedings 21st international conference on medical image computing and computer-assisted intervention–MICCAI 2018, Granada, Spain, 16–20 September 2018, Springer.

  57. Xue J, Zartarian V, Tulve N, Moya J, Freeman N, Auyeung W, et al. A meta-analysis of children’s object-to-mouth frequency data for estimating non-dietary ingestion exposure. J Expo Sci Environ Epidemiol. 2010;20:536–45.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the parents and their children who participated in this study. We are grateful to our student research assistants, Tionna Tolefree and Sofia Harrison, who visited participants’ homes and recorded the video data.

Funding

This project was supported by a grant from the US Environmental Protection Agency: Estimating Children’s Soil and Dust Ingestion Rates for Exposure Science EPA-G2020-STAR-D1. S.N.L. also received financial support from the National Institute of Environmental Health Sciences (NIEHS, Grant ID P30ES032756).

Author information

Authors and Affiliations

  1. Department of Environmental Health & Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Sara N. Lupolt, Qinfan Lyu, Christina Huynh & Keeve E. Nachman

  2. Risk Sciences and Public Policy Institute, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Sara N. Lupolt & Keeve E. Nachman

  3. Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA

    Guofeng Zhang, Jiahao Wang, Qihao Liu, Jiawei Peng, Xingrui Wang, Junjie Oscar Yin, Xiaoding Yuan, Yi Zhang & Alan L. Yuille

  4. Department of Pediatrics, Johns Hopkins School of Medicine, Baltimore, MD, USA

    Stacey Tang & Kristin M. Voegtline

  5. Department of Neuroscience, Johns Hopkins Krieger School of Arts and Sciences, Baltimore, MD, USA

    Jamie Cho

  6. Department of Cognitive Science, Johns Hopkins Krieger School of Arts and Sciences, Baltimore, MD, USA

    Alan L. Yuille

  7. Department of Obstetrics and Gynecology, Weill Cornell Medicine, New York, NY, USA

    Kristin M. Voegtline

  8. Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA

    Kristin M. Voegtline

  9. Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Keeve E. Nachman

Authors
  1. Sara N. Lupolt
    View author publications

    Search author on:PubMed Google Scholar

  2. Guofeng Zhang
    View author publications

    Search author on:PubMed Google Scholar

  3. Jiahao Wang
    View author publications

    Search author on:PubMed Google Scholar

  4. Stacey Tang
    View author publications

    Search author on:PubMed Google Scholar

  5. Qinfan Lyu
    View author publications

    Search author on:PubMed Google Scholar

  6. Jamie Cho
    View author publications

    Search author on:PubMed Google Scholar

  7. Christina Huynh
    View author publications

    Search author on:PubMed Google Scholar

  8. Qihao Liu
    View author publications

    Search author on:PubMed Google Scholar

  9. Jiawei Peng
    View author publications

    Search author on:PubMed Google Scholar

  10. Xingrui Wang
    View author publications

    Search author on:PubMed Google Scholar

  11. Junjie Oscar Yin
    View author publications

    Search author on:PubMed Google Scholar

  12. Xiaoding Yuan
    View author publications

    Search author on:PubMed Google Scholar

  13. Yi Zhang
    View author publications

    Search author on:PubMed Google Scholar

  14. Alan L. Yuille
    View author publications

    Search author on:PubMed Google Scholar

  15. Kristin M. Voegtline
    View author publications

    Search author on:PubMed Google Scholar

  16. Keeve E. Nachman
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Sara Lupolt: Writing—original draft, conceptualization, methodology, supervision, funding acquisition, project administration. Guofeng Zhang: Methodology, investigation, data analysis, writing—review and editing. Jiahao Wang: Methodology, investigation, data analysis, writing—review and editing. Stacey Tang: Methodology, data analysis, writing- review and editing. Jamie Cho: Methodology, data analysis. Qinfan Lyu: Data collection, data analysis, writing—review and editing. Christina Huynh: Recruitment, data collection, writing—review and editing. Qihao Liu: Methodology, data collection. Jiawei Peng: Methodology, data collection. Xingrui Wang: Methodology, data collection. Junjie Oscar Yin: Methodology, data collection. Xiaoding Yuan: Methodology, data collection. Yi Zhang: Methodology, data collection. Alan Yuille: Conceptualization, methodology, funding acquisition, supervision, writing—review and editing. Kristin Voegtline: Conceptualization, methodology, funding acquisition, supervision, writing- review and editing. Keeve E. Nachman: Conceptualization, methodology, writing- review and editing, funding acquisition, supervision, project administration.

Corresponding authors

Correspondence to Sara N. Lupolt or Keeve E. Nachman.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This study has been approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB00020023). All methods were performed in accordance with relevant guidelines and regulations.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lupolt, S.N., Zhang, G., Wang, J. et al. Development and evaluation of a computer vision algorithm for quantification of children’s microactivities. J Expo Sci Environ Epidemiol (2025). https://doi.org/10.1038/s41370-025-00814-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1038/s41370-025-00814-x

Keywords

Search

Advanced search

Quick links