Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Deep learning of HIV field-based rapid tests

Nature Medicine volume 27, pages 1165–1170 (2021)Cite this article

Subjects

Abstract

Although deep learning algorithms show increasing promise for disease diagnosis, their use with rapid diagnostic tests performed in the field has not been extensively tested. Here we use deep learning to classify images of rapid human immunodeficiency virus (HIV) tests acquired in rural South Africa. Using newly developed image capture protocols with the Samsung SM-P585 tablet, 60 fieldworkers routinely collected images of HIV lateral flow tests. From a library of 11,374 images, deep learning algorithms were trained to classify tests as positive or negative. A pilot field study of the algorithms deployed as a mobile application demonstrated high levels of sensitivity (97.8%) and specificity (100%) compared with traditional visual interpretation by humans—experienced nurses and newly trained community health worker staff—and reduced the number of false positives and false negatives. Our findings lay the foundations for a new paradigm of deep learning–enabled diagnostics in low- and middle-income countries, termed REASSURED diagnostics1, an acronym for real-time connectivity, ease of specimen collection, affordable, sensitive, specific, user-friendly, rapid, equipment-free and deliverable. Such diagnostics have the potential to provide a platform for workforce training, quality assurance, decision support and mobile connectivity to inform disease control strategies, strengthen healthcare system efficiency and improve patient outcomes and outbreak management in emerging infections.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Infographic illustrating the benefits of data capture in supporting field decisions.
Fig. 2: Standardization of image capture, image preprocessing and training library.
Fig. 3: Algorithm training and performance.
Fig. 4: Performance evaluation of our mHealth system compared to traditional visual interpretation: field pilot study.

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the AHRI data repository https://doi.org/10.23664/AHRI.M-AFRICA.2019.V1.

Code availability

Custom code used in this study is available at the public repository https://xip.uclb.com/product/classify_ai.

References

  1. Land, K. J., Boeras, D. I., Chen, X.-S., Ramsay, A. R. & Peeling, R. W. REASSURED diagnostics to inform disease control strategies, strengthen health systems and improve patient outcomes. Nat. Microbiol. 4, 46–54 (2019).

    Article  CAS  Google Scholar 

  2. Second WHO Model List of Essential In Vitro Diagnostics (WHO, 2019).

  3. Peeling, R. W. Diagnostics in a digital age: an opportunity to strengthen health systems and improve health outcomes. Int. Health 7, 384–389 (2015).

  4. Ghani, A. C., Burgess, D. H., Reynolds, A. & Rousseau, C. Expanding the role of diagnostic and prognostic tools for infectious diseases in resource-poor settings. Nature 528, S50–S52 (2015).

    Article  Google Scholar 

  5. Figueroa, C. et al. Reliability of HIV rapid diagnostic tests for self-testing compared with testing by health-care workers: a systematic review and meta-analysis. Lancet HIV 5, e277–e290 (2018).

    Article  Google Scholar 

  6. Klarkowski, D. B. et al. The evaluation of a rapid in situ HIV confirmation test in a programme with a high failure rate of the WHO HIV two-test diagnostic algorithm. PLoS ONE 4, e4351 (2009).

    Article  Google Scholar 

  7. Gray, R. H. et al. Limitations of rapid HIV-1 tests during screening for trials in Uganda: diagnostic test accuracy study. Brit. Med. J. 335, 188 (2007).

    Article  Google Scholar 

  8. Martin, E. G., Salaru, G., Paul, S. M. & Cadoff, E. M. Use of a rapid HIV testing algorithm to improve linkage to care. J. Clin. Virol. 52, S11–S15 (2011).

    Article  Google Scholar 

  9. Cham, F. et al. The World Health Organization African region external quality assessment scheme for anti-HIV serology. Afr. J. Lab. Med. 1, 39 (2012).

    PubMed  PubMed Central  Google Scholar 

  10. Galiwango, R. M. et al. Evaluation of current rapid HIV test algorithms in Rakai, Uganda. J. Virol. Methods 192, 25–27 (2013).

    Article  CAS  Google Scholar 

  11. Louis, F. J. et al. Evaluation of an external quality assessment program for HIV testing in Haiti, 2006–2011. Am. J. Clin. Pathol. 140, 867–871 (2013).

    Article  Google Scholar 

  12. Peck, R. B. et al. What should the ideal HIV self-test look like? A usability study of test prototypes in unsupervised HIV self-testing in Kenya, Malawi, and South Africa. AIDS Behav. 18, 422–432 (2014).

    Article  Google Scholar 

  13. Baveewo, S. et al. Potential for false positive HIV test results with the serial rapid HIV testing algorithm. BMC Res. Notes 5, 154 (2012).

    Article  Google Scholar 

  14. Crucitti, T., Taylor, D., Beelaert, G., Fransen, K. & Van Damme, L. Performance of a rapid and simple HIV testing algorithm in a multicenter phase III microbicide clinical trial. Clin. Vaccine Immunol. 18, 1480–1485 (2011).

    Article  Google Scholar 

  15. Tegbaru, B. et al. Assessment of the implementation of HIV-rapid test kits at different levels of health institutions in Ethiopia. Ethiop. Med. J. 45, 293–299 (2007).

    PubMed  Google Scholar 

  16. Johnson, C. C. et al. To err is human, to correct is public health: a systematic review examining poor quality testing and misdiagnosis of HIV status. J. Int. AIDS Soc. 20, 21755 (2017).

    Article  Google Scholar 

  17. Learmonth, K. M. et al. Assessing proficiency of interpretation of rapid human immunodeficiency virus assays in nonlaboratory settings: ensuring quality of testing. J. Clin. Microbiol. 46, 1692–1697 (2008).

    Article  Google Scholar 

  18. García, P. J. et al. Rapid syphilis tests as catalysts for health systems strengthening: a case study from Peru. PLoS ONE 8, e66905 (2013).

    Article  Google Scholar 

  19. Sacks, R., Omodele-Lucien, A., Whitbread, N., Muir, D. & Smith, A. Rapid HIV testing using DetermineTM HIV 1/2 antibody tests: is there a difference between the visual appearance of true- and false-positive tests? Int. J. STD AIDS 23, 644–646 (2012).

    Article  CAS  Google Scholar 

  20. Doan, M. & Carpenter, A. E. Leveraging machine vision in cell-based diagnostics to do more with less. Nat. Mater. 18, 414–418 (2019).

    Article  CAS  Google Scholar 

  21. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  CAS  Google Scholar 

  22. De Fauw, J. et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med. 24, 1342–1350 (2018).

    Article  Google Scholar 

  23. Xu, Y. et al. Deep learning predicts lung cancer treatment response from serial medical imaging. Clin. Cancer Res. 25, 3266–3275 (2019).

    Article  Google Scholar 

  24. Silver, D. et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140–1144 (2018).

    Article  CAS  Google Scholar 

  25. Ascent of machine learning in medicine. Nat. Mater. 18, 407 (2019).

  26. Ching, T. et al. Opportunities and obstacles for deep learning in biology and medicine. J. R. Soc. Interface 15, 20170387 (2018).

  27. Zeng, N., Wang, Z., Zhang, H., Liu, W. & Alsaadi, F. E. Deep belief networks for quantitative analysis of a gold immunochromatographic strip. Cogn. Comput. 8, 684–692 (2016).

    Article  Google Scholar 

  28. Carrio, A., Sampedro, C., Sanchez-Lopez, J. L., Pimienta, M. & Campoy, P. Automated low-cost smartphone-based lateral flow saliva test reader for drugs-of-abuse detection. Sensors (Basel) 15, 29569–29593 (2015).

    Article  Google Scholar 

  29. Neuman, M. et al. The effectiveness and cost-effectiveness of community-based lay distribution of HIV self-tests in increasing uptake of HIV testing among adults in rural Malawi and rural and peri-urban Zambia: protocol for STAR (self-testing for Africa) cluster randomized evaluations. BMC Public Health 18, 1234 (2018).

    Article  Google Scholar 

  30. Aicken, C. R. H. et al. Young people’s perceptions of smartphone-enabled self-testing and online care for sexually transmitted infections: qualitative interview study. BMC Public Health 16, 974 (2016).

    Article  Google Scholar 

  31. Witzel, T. C., Weatherburn, P., Rodger, A. J., Bourne, A. H. & Burns, F. M. Risk, reassurance and routine: a qualitative study of narrative understandings of the potential for HIV self-testing among men who have sex with men in England. BMC Public Health 17, 491 (2017).

    Article  Google Scholar 

  32. Nsabimana, A. P. et al. Bringing real-time geospatial precision to HIV surveillance through smartphones: feasibility study. JMIR Public Health Surveill. 4, e11203 (2018).

    Article  Google Scholar 

  33. Laksanasopin, T. et al. A smartphone dongle for diagnosis of infectious diseases at the point of care. Sci. Transl. Med. 7, 273re1 (2015).

    Article  Google Scholar 

  34. Mudanyali, O. et al. Integrated rapid-diagnostic-test reader platform on a cellphone. Lab Chip 12, 2678–2686 (2012).

    Article  Google Scholar 

  35. Allan-Blitz, L.-T. et al. Field evaluation of a smartphone-based electronic reader of rapid dual HIV and syphilis point-of-care immunoassays. Sex. Transm. Infect. 94, 589–593 (2018).

    Article  Google Scholar 

  36. Feng, S. et al. Immunochromatographic diagnostic test analysis using Google Glass. ACS Nano 8, 3069–3079 (2014).

  37. Guan, Q. et al. Diagnose like a radiologist: attention guided convolutional neural network for thorax disease classification. Preprint at https://arxiv.org/abs/1801.09927 (2018).

  38. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. & Chen, L.-C. MobileNetV2: inverted residuals and linear bottlenecks. Preprint at https://arxiv.org/abs/1801.04381 (2018).

  39. Chaturvedi, S. S., Gupta, K. & Prasad, P. S. Skin lesion analyser: an efficient seven-way multi-class skin cancer classification using MobileNet. In International Conference on Advanced Machine Learning Technologies and Applications 165–176 (Springer, 2021).

  40. Howard, A. et al. Searching for MobileNetV3. Preprint at https://arxiv.org/abs/1905.02244 (2019).

  41. Gareta, D. et al. Cohort profile update: Africa Centre Demographic Information System (ACDIS) and population-based HIV survey. Int. J. Epidemiol. 50, 33 (2021).

    Article  Google Scholar 

Download references

Acknowledgements

We thank the community of the uMkhanyakude district and the study participants, as well as the AHRI team of fieldworkers and their supervisors. We thank A. Koza, Z. Thabethe, T. Madini, N. Okesola and S. Msane for their help with the pilot study; D. Gareta and J. Dreyer for IT support; V. Lampos and I. J. Cox for useful discussions; and E. Manning and J. McHugh for their help with editing and project management. This research was funded by the m-Africa Medical Research Council GCRF Global Infections Foundation Award (no. MR/P024378/1, to C.H., D.P., K.H., M.S., R.A.M. and V.T.) and is part of the EDCTP2 program supported by the European Union, i-sense Engineering and Physical Sciences Research Council Interdisciplinary Research Collaboration (EPSRC IRC) in Early Warning Sensing Systems for Infectious Disease (no. EP/K031953/1, to R.A.M., V.T., D.P., S.M., S.G., N.A. and M.S.), the i-sense: EPSRC IRC in Agile Early Warning Sensing Systems for Infectious Diseases and Antimicrobial Resistance (no. EP/R00529X/1, to R.A.M., V.T., D.P., S.G., N.A. and S.M.) and supported by the National Institute for Health Research University College London Hospitals Biomedical Research Centre (R.A.M. and S.M.). We thank the m-Africa and i-sense investigators and advisory boards. The AHRI is supported by core funding from the Wellcome Trust (core grant no. 082384/Z/07/Z, to T.S., D.P. and K.H.).

Author information

Authors and Affiliations

  1. London Centre for Nanotechnology, University College London, London, UK

    Valérian Turbé, Sepehr Meshkinfamfard, Jobie Budd, Nestor Arsenov & Rachel A. McKendry

  2. Africa Health Research Institute, Nelson R. Mandela Medical School, Durban, South Africa

    Carina Herbst, Thobeka Mngomezulu, Nondumiso Dlamini, Thembani Mhlongo, Theresa Smit, Deenan Pillay, Kobus Herbst & Maryam Shahmanesh

  3. Department of Computer Science, University College London, London, UK

    Valeriia Cherepanova & Koki Shimada

  4. Division of Medicine, University College London, London, UK

    Jobie Budd & Rachel A. McKendry

  5. UCL Centre for Advanced Spatial Analysis, London, UK

    Steven Gray

  6. Division of Infection and Immunity, University College London, London, UK

    Deenan Pillay

  7. DSI-MRC South African Population Research Infrastructure Network, Durban, South Africa

    Kobus Herbst

  8. Institute for Global Health, University College London, London, UK

    Maryam Shahmanesh

Authors
  1. Valérian Turbé
    View author publications

    Search author on:PubMed Google Scholar

  2. Carina Herbst
    View author publications

    Search author on:PubMed Google Scholar

  3. Thobeka Mngomezulu
    View author publications

    Search author on:PubMed Google Scholar

  4. Sepehr Meshkinfamfard
    View author publications

    Search author on:PubMed Google Scholar

  5. Nondumiso Dlamini
    View author publications

    Search author on:PubMed Google Scholar

  6. Thembani Mhlongo
    View author publications

    Search author on:PubMed Google Scholar

  7. Theresa Smit
    View author publications

    Search author on:PubMed Google Scholar

  8. Valeriia Cherepanova
    View author publications

    Search author on:PubMed Google Scholar

  9. Koki Shimada
    View author publications

    Search author on:PubMed Google Scholar

  10. Jobie Budd
    View author publications

    Search author on:PubMed Google Scholar

  11. Nestor Arsenov
    View author publications

    Search author on:PubMed Google Scholar

  12. Steven Gray
    View author publications

    Search author on:PubMed Google Scholar

  13. Deenan Pillay
    View author publications

    Search author on:PubMed Google Scholar

  14. Kobus Herbst
    View author publications

    Search author on:PubMed Google Scholar

  15. Maryam Shahmanesh
    View author publications

    Search author on:PubMed Google Scholar

  16. Rachel A. McKendry
    View author publications

    Search author on:PubMed Google Scholar

Contributions

V.T. and R.A.M. wrote the manuscript with input from coauthors. V.T., C.H., T. Mngomezulu, N.D. and T. Mhlongo collected field data. V.T. and S.M. developed the machine learning models with contributions from V.C., K.S., S.G. and R.A.M. V.T., N.A. and J.B. were involved in manual data preprocessing. K.H. oversaw data collection and management. T.S. and M.S. provided access to anonymized blood samples used in the pilot study. R.A.M., V.T., M.S., K.H. and D.P. conceived the overall project, designed the study and secured funding. R.A.M. was the principal investigator with overall responsibility for the i-sense EPSRC IRC and m-Africa programs, and was supervisor of the research associates (V.T., S.M. and N.A.) and students (V.C., K.S. and J.B.) involved in this study.

Corresponding authors

Correspondence to Valérian Turbé, Kobus Herbst, Maryam Shahmanesh or Rachel A. McKendry.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Medicine thanks Nicholas Durr and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Michael Basson was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Standard Operating Procedure for HIV RDT image collection.

Document used for training and distributed to all AHRI fieldworkers involved in data collection. Left-hand side: example of valid and invalid photographs. Right-hand side: step-by-step guidelines for capturing pictures of HIV RDTs.

Extended Data Fig. 2 Screenshots of the Android application, to illustrate the capture of the HIV RDT image at the time of reading the test result.

Images were captured sequentially from left to right. The end user is asked to align the test with the overlay on the screen, then continuously press the capture button for 3 seconds, after which the image is automatically captured and processed to extract the ROI. The 3 seconds press feature was implemented as a result of consultation with end users in the optimisation phase of the app development.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Turbé, V., Herbst, C., Mngomezulu, T. et al. Deep learning of HIV field-based rapid tests. Nat Med 27, 1165–1170 (2021). https://doi.org/10.1038/s41591-021-01384-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41591-021-01384-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing