Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Predicting firearm suicide among US Army veterans transitioning from active service

Abstract

US veterans are significantly more likely than civilians to die by suicide. Machine-learning models have been developed to target high-risk transitioning service members for suicide prevention interventions to reduce veteran suicides. These models are suicide method-agnostic. However, firearms are involved in most veteran suicides, and firearm-specific preventions exist. We used data from US Army veterans from 2010 to 2019 (N = 800,579) to develop and compare firearm-specific machine-learning models with a method-agnostic model to predict firearm suicides among transitioning Army veterans up to 10 years after discharge. The models performed comparably overall (area under the receiver operating characteristic curve = 0.710–0.708; integrated calibration index = 0.0003–0.0005% for firearm-specific and method-agnostic models, respectively), with the best model depending on the intervention threshold. Results from this study show the method-agnostic model was better at predicting firearm suicides at the highest intervention threshold, whereas the firearm-specific model was better at lower thresholds. When considering fairness with respect to sex and race/ethnicity, the firearm-specific model was best across all thresholds. Thus, model choice depends on weighing numerous factors, and optimal thresholds might differ for coordinated firearm-specific and method-agnostic interventions.

This is a preview of subscription content, access via your institution

Access options

Buy this article

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Decision curves for net benefit across models.
Fig. 2: Predictor importance based on SHAP values of the ensemble firearm-specific model.

Similar content being viewed by others

Data availability

STARRS-LS Wave 1, Wave 2 and Wave 3 data, as well as data from the earlier Army STARRS New Soldier Study (NSS), All Army Study (AAS) and Pre and Post Deployment studies (PPDS), are available through the Inter-university Consortium for Political and Social Research (ICPSR) at the University of Michigan (https://www.icpsr.umich.edu/web/ICPSR/studies/35197). STARRS-LS and Army STARRS data are restricted from general dissemination, meaning that a confidential data use agreement must be established before access. Researchers interested in gaining access to the data can submit their applications via ICPSR’s online Restricted Contracting System. Guidelines for applying for access to this data can be found under the data and documentation tab at the above URL. The STARRS Historical Administrative Data Study (HADS) data are not available for public release.

Code availability

Code used to produce the statistical output can be located via https://osf.io/csm59/. Because the data used in this study are not publicly available and require DoD clearance to access, we cannot post the source code used to read in the Army administrative data (which requires DoD clearance) and create the 1,700+ variables used in this analysis. However, we have provided detailed appendices about each of the variables used as covariates in the analyses and have provided here the statistical code for generating the tables and figures.

References

  1. WISQARS Fatal and Nonfatal Injury Reports (Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 2024); https://wisqars.cdc.gov/reports/

  2. Department of Defense Annual Report on Suicide in the Military: Calendar Year 2021 Vol. 2024 (Department of Defense, 2022); https://www.dspo.mil/Portals/113/Documents/2022%20ASR/FY21%20ASR.pdf?ver=soZ94xt2yM905wj9TbwI3g%3d%3d%E2%80%8B

  3. 2023 National Veteran Suicide Prevention Annual Report (US Department of Veterans Affairs, 2023); https://www.mentalhealth.va.gov/docs/data-sheets/2023/2023-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-508.pdf

  4. Ravindran, C., Morley, S. W., Stephens, B. M., Stanley, I. H. & Reger, M. A. Association of suicide risk with transition to civilian life among US Military service members. JAMA Netw. Open 3, e2016261 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Shen, Y.-C., Cunha, J. M. & Williams, T. V. Time-varying associations of suicide with deployments, mental health conditions, and stressful life events among current and former US Military personnel: a retrospective multivariate analysis. Lancet Psychiatry 3, 1039–1048 (2016).

    Article  PubMed  Google Scholar 

  6. VA/DOD Clinical Practice Guideline for the Assessment and Management of Patients at Risk for Suicide Version 3.0 (US Department of Veterans Affairs, 2024); https://www.healthquality.va.gov/guidelines/mh/srb/

  7. Veterans Transitioning from Service (US Department of Veterans Affairs, 2024); https://www.mentalhealth.va.gov/transitioning-service/resources.asp

  8. Nock, M. K., Ramirez, F. & Rankin, O. Advancing our understanding of the who, when, and why of suicide risk. JAMA Psychiatry 76, 11–12 (2019).

    Article  PubMed  Google Scholar 

  9. Kennedy, C. J. et al. Predicting suicides among US Army soldiers after leaving active service. JAMA Psychiatry 18, 1215–1224 (2024).

    Article  Google Scholar 

  10. Kearns, J. C. et al. A practical risk calculator for suicidal behavior among transitioning US Army soldiers: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS). Psychol. Med. 53, 7096–7105 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Secretary of Defense, Secretary of Veterans Affairs & Secretary of Homeland Security. Joint Action Plan for Supporting Veterans During their Transition from Uniformed Service to Civilian Life (US Department of Veterans Affairs, 2018); https://www.va.gov/opa/docs/joint-action-plan-05-03-18.pdf

  12. Those Who Serve: Addressing Firearm Suicide Among Our Nation’s Veterans (Everytown Research & Policy, 2024); https://everytownresearch.org/report/those-who-serve/

  13. Kaczkowski, W. et al. Notes from the field: firearm suicide rates, by race and ethnicity—United States, 2019–2022. MMWR Morb. Mortal. Wkly. Rep. 72, 1307–1308 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Stanley, I. H., Embrey, E. P. & Bebarta, V. S. Actualizing military suicide prevention through digital health modernization. JAMA Psychiatry 81, 1173–1174 (2024).

    Article  PubMed  Google Scholar 

  15. Raines, A. M. et al. Comparing the interactions of risk factors by method of suicide among veterans: a moderated network analysis approach. J. Psychopathol. Clin. Sci. 133, 273–284 (2024).

    Article  PubMed  Google Scholar 

  16. Ammerman, B. A. & Reger, M. A. Evaluation of prevention efforts and risk factors among veteran suicide decedents who died by firearm. Suicide Life Threat. Behav. 50, 679–687 (2020).

    Article  PubMed  Google Scholar 

  17. LeardMann, C. A. et al. Prospective comparison of risk factors for firearm suicide and non-firearm suicide in a large population-based cohort of current and former US service members: findings from the Millennium Cohort Study. Lancet Reg. Health Am. 36, 100802 (2024).

    PubMed  PubMed Central  Google Scholar 

  18. Vickers, A. J., van Calster, B. & Steyerberg, E. W. A simple, step-by-step guide to interpreting decision curve analysis. Diagn. Progn. Res. 3, 18 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. In Proc. 31st International Conference on Neural Information Processing Systems 4768–4777 (Curran Associates, 2017); https://dl.acm.org/doi/10.5555/3295222.3295230

  20. Kaufman, E. J., Song, J., Xiong, R., Seamon, M. J. & Delgado, M. K. Fatal and nonfatal firearm injury rates by race and ethnicity in the United States, 2019 to 2020. Ann. Intern. Med. 177, 1157–1169 (2024).

    Article  PubMed  Google Scholar 

  21. Paashaus, L. et al. From decision to action: suicidal history and time between decision to die and actual suicide attempt. Clin. Psychol. Psychother. 28, 1427–1434 (2021).

    Article  PubMed  Google Scholar 

  22. Cai, Z., Junus, A., Chang, Q. & Yip, P. S. The lethality of suicide methods: a systematic review and meta-analysis. J. Affect. Disord. 300, 121–129 (2022).

    Article  PubMed  Google Scholar 

  23. Constans, J. I., Houtsma, C., Bailey, M. & True, G. The Armory Project: partnering with firearm retailers to promote and provide voluntary out-of-home firearm storage. Suicide Life Threat. Behav. 53, 716–724 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Kelly, T., Brandspigel, S., Polzer, E. & Betz, M. E. Firearm storage maps: a pragmatic approach to reduce firearm suicide during times of risk. Ann. Intern. Med. 172, 351–353 (2020).

    Article  PubMed  Google Scholar 

  25. Betz, M. E., Barnard, L. M., Knoepke, C. E., Rivara, F. P. & Rowhani-Rahbar, A. Creation of online maps for voluntary out-of-home firearm storage: experiences and opportunities. Prev. Med. Rep. 32, 102167 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Extreme Risk Protection Orders (Giffords Law Center, 2024); https://giffords.org/lawcenter/gun-laws/policy-areas/who-can-have-a-gun/extreme-risk-protection-orders/

  27. Swanson, J. W. et al. Suicide prevention effects of extreme risk protection order laws in four states. J. Am. Acad. Psychiatry Law 52, 327–337 (2024).

    PubMed  Google Scholar 

  28. Miller, M., Zhang, Y., Studdert, D. M. & Swanson, S. Updated estimate of the number of extreme risk protection orders needed to prevent 1 suicide. JAMA Netw. Open 7, e2414864 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Zeoli, A. M., Frattaroli, S., Aassar, M., Cooper, C. E. & Omaki, E. Extreme risk protection orders in the United States: a systematic review of the research. Annu. Rev. Criminol. 8, 485–504 (2025).

    Article  Google Scholar 

  30. Kravitz-Wirtz, N., Aubel, A. J., Pallin, R. & Wintemute, G. J. Public awareness of and personal willingness to use California’s Extreme Risk Protection Order Law to prevent firearm-related harm. JAMA Health Forum 2, e210975 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Dempsey, C. L. et al. Association of firearm ownership, use, accessibility, and storage practices with suicide risk among US Army soldiers. JAMA Netw. Open 2, e195383 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Barber, C. W. & Miller, M. J. Reducing a suicidal person’s access to lethal means of suicide: a research agenda. Am. J. Prev. Med. 47, S264–S272 (2014).

    Article  PubMed  Google Scholar 

  33. Pallin, R., Spitzer, S. A., Ranney, M. L., Betz, M. E. & Wintemute, G. J. Preventing firearm-related death and injury. Ann. Intern. Med. 170, ITC81–ITC96 (2019).

    Article  PubMed  Google Scholar 

  34. Anestis, M. D., Bryan, C. J., Capron, D. W. & Bryan, A. O. Lethal means counseling, distribution of cable locks, and safe firearm storage practices among the Mississippi National Guard: a factorial randomized controlled trial, 2018–2020. Am. J. Public Health 111, 309–317 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Simonetti, J. A., Rowhani-Rahbar, A., King, C., Bennett, E. & Rivara, F. P. Evaluation of a community-based safe firearm and ammunition storage intervention. Inj. Prev. 24, 218–223 (2018).

    Article  PubMed  Google Scholar 

  36. National Veteran Suicide Prevention Annual Report (US Department of Veterans Affairs, 2023); https://www.mentalhealth.va.gov/docs/data-sheets/2023/2023-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-508.pdf

  37. Stanley, I. H. et al. Project Safeguard: challenges and opportunities of a universal rollout of peer-delivered lethal means safety counseling at a US Military installation. Suicide Life Threat. Behav. 54, 489–500 (2024).

    Article  PubMed  Google Scholar 

  38. Houtsma, C., Hohman, M., Anestis, M. D., Bryan, C. J. & True, G. Development of a peer-delivered lethal means counseling intervention for firearm owning veterans: peer engagement and exploration of responsibility and safety (PEERS). Suicide Life Threat. Behav. 55, e13138 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Shenassa, E. D., Rogers, M. L., Spalding, K. L. & Roberts, M. B. Safer storage of firearms at home and risk of suicide: a study of protective factors in a nationally representative sample. J. Epidemiol. Community Health 58, 841–848 (2004).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Betz, M. E. et al. Firearm suicide prevention in the US Military: recommendations from a national summit. Mil. Med. 188, 231–235 (2023).

    Article  PubMed  Google Scholar 

  41. National Defense Authorization Act for Fiscal Year 2011. Pub. L. No. 111-383 (US Congress, 2011); https://www.congress.gov/bill/118th-congress/senate-bill/4638

  42. Haviland, M. J. et al. Assessment of county-level proxy variables for household firearm ownership. Prev. Med. 148, 106571 (2021).

    Article  PubMed  Google Scholar 

  43. Kessler, R. C. et al. Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Int. J. Methods Psychiatr. Res. 22, 267–275 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod) the tripod statement. Circulation 131, 211–219 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Efron, B. Logistic regression, survival analysis, and the Kaplan-Meier curve. J. Am. Stat. Assoc. 83, 414–425 (1988).

    Article  Google Scholar 

  46. Bansal, A. & Pepe, M. S. Estimating improvement in prediction with matched case-control designs. Lifetime Data Anal. 19, 170–201 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Jiang, T. et al. Using machine learning to predict suicide in the 30 days after discharge from psychiatric hospital in Denmark. Br. J. Psychiatry 219, 440–447 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Saxe, G. N., Ma, S., Ren, J. & Aliferis, C. Machine learning methods to predict child posttraumatic stress: a proof of concept study. BMC Psychiatry 17, 223 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Polley, E. C., LeDell, E., Kennedy, C., Lendle, S. & van der Laan, M. J. SuperLearner: super learner prediction (R package). GitHub https://github.com/ecpolley/SuperLearner (2024).

  50. Polley, E. C., Rose S., & van der Laan M. J. in Targeted Learning: Causal Inference for Observational and Experimental Data (eds Rose, S. & van der Laan, M. J.) 43–66 (Springer, 2011).

  51. LeDell, E., van der Laan, M. J. & Petersen, M. AUC-maximizing ensembles through metalearning. Int. J. Biostat. 12, 203–218 (2016).

    Article  PubMed  Google Scholar 

  52. Kabir, M.F. & Ludwig, S.A. Enhancing the performance of classification using super learning. Data-Enabled Discov. Appl. 3, 5 (2019).

    Article  Google Scholar 

  53. Kennedy, C. Guide to SuperLearner (University of California, 2017); https://cran.r-project.org/web/packages/SuperLearner/vignettes/Guide-to-SuperLearner.html

  54. Wright, M. N. & Ziegler, A. Ranger: a fast implementation of random forests for high dimensional data in C++ and r. J. Stat. Softw. https://doi.org/10.18637/jss.v077.i01 (2017).

    Article  Google Scholar 

  55. Niculescu-Mizil, A. & Caruana, R. Predicting good probabilities with supervised learning. In Proc. 22nd International Conference on Machine Learning 625–632 (Association for Computing Machinery, 2005); https://doi.org/10.1145/1102351.1102430

  56. Jiang, X., Osl, M., Kim, J. & Ohno-Machado, L. Calibrating predictive model estimates to support personalized medicine. J. Am. Med. Inform. Assoc. 19, 263–274 (2012).

    Article  PubMed  Google Scholar 

  57. Hanley, J. A. & McNeil, B. J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 29–36 (1982).

    Article  PubMed  Google Scholar 

  58. Davis, J. & Goadrich, M. The relationship between precision-recall and ROC curves. In Proc. 23rd International Conference on Machine Learning 233–240 (Association for Computing Machinery, 2006); https://doi.org/10.1145/1143844.1143874

  59. Austin, P. C. & Steyerberg, E. W. The integrated calibration index (ICI) and related metrics for quantifying the calibration of logistic regression models. Stat. Med. 38, 4051–4065 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Austin, P. C. & Steyerberg, E. W. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat. Med. 33, 517–535 (2014).

    Article  PubMed  Google Scholar 

  61. Vickers, A.J., Van Calster, B. & Steyerberg, E.W. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. Br. Med. J. 352, i6 (2016).

    Article  Google Scholar 

  62. Vickers, A. J. & Elkin, E. B. Decision curve analysis: a novel method for evaluating prediction models. Med. Decis. Making 26, 565–574 (2006).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Van Calster, B. et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur. Urol. 74, 796–804 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  64. Barocas, S., Hardt, M. & Narayanan, A. Fairness and Machine Learning: Limitations and Opportunities (MIT Press, 2023); https://fairmlbook.org/

  65. Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2, 56–67 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  66. SAS v9.4 (SAS Institute, 2013); https://support.sas.com/software/94/

  67. R Core Team: The R Project for Statistical Computing (R Foundation, 2021); https://www.r-project.org/

Download references

Acknowledgements

The Army STARRS Team consists of co-principal investigators R. J. Ursano (Uniformed Services University) and M. B. Stein (University of California San Diego and VA San Diego Healthcare System); site principal investigators J. Wagner (University of Michigan) and R. C. Kessler (Harvard Medical School); Army scientific consultant/liaison K. Cox (Office of the Assistant Secretary of the Army (Manpower and Reserve Affairs)). Other team members are P. A. Aliaga (Uniformed Services University); D. M. Benedek (Uniformed Services University); L. Campbell-Sills (University of California San Diego); C. L. Dempsey (Uniformed Services University); C. S. Fullerton (Uniformed Services University); N. Gebler (University of Michigan); M. House (University of Michigan); P. E. Hurwitz (Uniformed Services University); S. Jain (University of California San Diego); T.-C. Kao (Uniformed Services University); C. J. Kennedy (Massachusetts General Hospital); L. Lewandowski-RompsD (University of Michigan); A. Luedtke (University of Washington and Fred Hutchinson Cancer Research Center); H. H. Mash (Uniformed Services University); J. A. Naifeh (Uniformed Services University); M. K. Nock (Harvard University); N. A. Sampson (Harvard Medical School); R. Shor (Uniformed Services University); and A. M. Zaslavsky (Harvard Medical School). C.H. and E.R.E. were supported in part by the United States Department of Veterans Affairs, Clinical Sciences Research and Development Service (CSR&D) VA-STARRS Researcher-in-Residence Program (project SPR-002-24F). C.J.K. was supported by the National Institute of Mental Health (K01MH135131). Army STARRS was sponsored by the Department of the Army and funded under cooperative agreement number U01MH087981 (2009–2015) with the National Institute of Mental Health (NIMH). Subsequently, STARRS-LS was sponsored and funded by the Department of Defense (USUHS grant number HU0001-15-2-0004). The contents are solely the responsibility of the authors and do not necessarily represent the views of the NIMH, the Department of the Army, the Department of Defense or the Department of Veteran Affairs.

Author information

Authors and Affiliations

Authors

Contributions

C.H. and R.C.K. were responsible for the study concept and design. C.J.K., H.L., N.A.S., J.C.G., B.P.M. and R.C.K. acquired, analyzed and interpreted the data. C.H., C.J.K., H.L. and R.C.K. were involved in drafting of the manuscript. C.J.K. and H.L. conducted the primary statistical analyses. B.P.M., J.W., M.B.S., R.J.U. and R.C.K. obtained funding. E.R.E., N.A.S., J.C.G., B.P.M., M.K.N., J.W., M.B.S., R.J.U. and R.C.K. provided technical, administrative and material support. C.J.K., N.A.S., J.C.G., B.P.M., M.K.N., J.W., M.B.S., R.J.U. and R.C.K. provided study supervision. All authors were involved in critical revision of the manuscript for important intellectual content and approved the final version for submission.

Corresponding author

Correspondence to Claire Houtsma.

Ethics declarations

Competing interests

In the past 3 years, R.C.K. was a consultant for Cambridge Health Alliance, Canandaigua VA Medical Center, Child Mind Institute, Holmusk, Massachusetts General Hospital, Partners Healthcare, Inc., RallyPoint Networks, Inc., Sage Therapeutics and University of North Carolina. He has stock options in Cerebral Inc., Mirah, PYM (Prepare Your Mind), Roga Sciences and Verisense Health. In the past 3 years, M.B.S. received consulting income from Actelion, Acadia Pharmaceuticals, Aptinyx, atai Life Sciences, Boehringer Ingelheim, Bionomics, BioXcel Therapeutics, Clexio, EmpowerPharm, Engrail Therapeutics, GW Pharmaceuticals, Janssen, Jazz Pharmaceuticals and Roche/Genentech. He has stock options in Oxeia Biopharmaceuticals and EpiVario. He is paid for his editorial work on Biological Psychiatry (deputy editor) and UpToDate (co-editor-in-chief for psychiatry). No other disclosures reported.

Peer review

Peer review information

Nature Mental Health thanks Jim Schmeckenbecher and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Fig. 1, Tables 1–6, Measures and References.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Houtsma, C., Kennedy, C.J., Liu, H. et al. Predicting firearm suicide among US Army veterans transitioning from active service. Nat. Mental Health 4, 125–135 (2026). https://doi.org/10.1038/s44220-025-00559-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s44220-025-00559-4

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing