Abstract
NarxCare®, a proprietary opioid risk scoring system embedded in Prescription Drug Monitoring Programs (PDMPs), has generated significant patient complaints. We adhered to the technical specifications and applied them to PDMP and IQVIA PharMetrics® Plus Closed Health Plan claims database. Despite adding socioeconomic covariates, precision (0.01–0.32) was far below the reported benchmark of 0.75, and F1 scores (0.02–0.39) were also substantially lower than the benchmark value of 0.65, across all our reconstructed models.
Data availability
The CURES dataset is available upon request from the Department of Justice. The census data can be obtained from the US Zip Codes Database (Pareto SoftwareTM, version 2023). Concerning access to and use of the IQVIA PharMetrics® Plus for Academics dataset, which is licensed to Chapman University under the terms of its agreement with IQVIA Inc.
Code availability
The code is publicly accessible at https://github.com/Sherry-Yun-Wang/Algorithmic-Opacity-in-Opioid-Risk-Scoring-Need-for-Transparent-AI-Regulation-in-Healthcare.
References
Ardeljan, L. D. et al. Current state of opioid stewardship. Am. J. Health-Syst. Pharm. 77, 636–643 (2020).
Bhagwat, A. M., Ferryman, K. S. & Gibbons, J. B. Mitigating algorithmic bias in opioid risk-score modeling to ensure equitable access to pain relief. Nat. Med. 29, 769–770 (2023).
Bamboo Health. NarxCare Application Overview, https://dopl.idaho.gov/wp-content/uploads/2024/03/BOP-PDMP-Overview-NarxCare.pdf (2023).
Larochelle, M. R. et al. Medication for opioid use disorder after nonfatal opioid overdose and association with mortality: a cohort study. Ann. Intern. Med. 169, 137–145 (2018).
Wakeman, S. E. et al. Comparative effectiveness of different treatment pathways for opioid use disorder. JAMA Netw. Open 3, e1920622 (2020).
Biondi, B. E., Zheng, X., Frank, C. A., Petrakis, I. & Springer, S. A. A literature review examining primary outcomes of medication treatment studies for opioid use disorder: what outcome should be used to measure opioid treatment success? Am. J. Addict. 29, 249–267 (2020).
Acharya, M. et al. Comparative study of opioid initiation with tramadol, short-acting hydrocodone, or short-acting oxycodone on opioid-related adverse outcomes among chronic noncancer pain patients. Clin. J. pain. 39, 107–118 (2023).
Xu, Z., Shen, D., Nie, T. & Kou, Y. A hybrid sampling algorithm combining M-SMOTE and ENN based on random forest for medical imbalanced data. J. Biomed. Inform. 107, 103465 (2020).
Buonora, M. J., Axson, S. A., Cohen, S. M. & Becker, W. C. Paths forward for clinicians amidst the rise of unregulated clinical decision support software: our perspective on NarxCare. J. Gen. Intern. Med. 39, 858–862 (2024).
Siegel, Z. In a World of Stigma and Bias, can a computer algorithm really predict overdose risk?: A machine-learning algorithm is being deployed across America to prevent overdose deaths. but could it be causing more pain?. Ann. Emerg. Med. 79, A16–A19 (2022).
Szalavitz, M. The pain was unbearable. So why did doctors turn her away. Wired. 11 https://www.wired.com/story/opioid-drug-addiction-algorithm-chronic-pain/ (2021).
Gottlieb, S. Congress Must Update FDA Regulations for Medical AI. JAMA Forum 5, e242691 (2024).
U.S. Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. U.S. Department of Health and Human Services. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software (2026).
Harvey, H. B. & Gowda, V. How the FDA regulates AI. Acad. Radiol. 27, 58–61 (2020).
Boubker, J. When medical devices have a mind of their own: the challenges of regulating artificial intelligence. Am. J. Law Med. 47, 427–454 (2021).
White House. Winning the AI Race: America’s AI Action Plan, https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf (2025).
SimpleMaps. United States ZIP Code Database (Version 2025), https://simplemaps.com/data/us-zip-codes (2025).
Li, C. et al. Realizing the potential of social determinants data in EHR systems: a scoping review of approaches for screening, linkage, extraction, analysis, and interventions. J. Clin. Transl. Sci. 8, e147 (2024).
Hurley, R. W. et al. Evidence-based framework for identifying opioid use disorder in administrative data: a systematic review and methodological development study. Pain Med. 27, 145–159 (2025).
Elkington, K. S. et al. Examining the impact of the innovative opioid court model on treatment access and court outcomes for court participants. J. Addict. Med. 18, 635–642 (2024).
Roy P.J. et al. Impact of study design decisions on identification of treatment initiators of medications for opioid use disorder. https://doi.org/10.1111/add.70288 (2025).
Acknowledgements
We extend our heartfelt gratitude to the California Department of Justice for their invaluable support in providing the data and their unwavering assistance throughout our research journey. We acknowledge that CURES is not associated with the NarxCare platform, and any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the California Department of Justice CURES Program or IQVIA Inc. This project was supported by funding from the National Institute of Health (NIH) AIM-AHEAD program.
Author information
Authors and Affiliations
Contributions
S.Y.W. conceptualized the research idea, designed the study, authored the primary manuscript, and secured funding as the Principal Investigator (PI). R. S. conducted the data analysis. A.L., C. Z., and X. H. contributed to the major revision. All authors edited,reviewed, and approved the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wang, S.Y., Stofer, R., Chu, Z. et al. Algorithmic opacity in opioid risk scoring and the need for transparent AI regulation. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-02491-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41746-026-02491-y
