Fig. 2: Balance performance of the regularized logistic regression-based propensity score models (LR-PS) selected by different model selection strategies, OneFlorida database, 2012–2020.
From: High-throughput target trial emulation for Alzheimer’s disease drug repurposing with real-world data
a The proportion of successfully balanced trials by LR-PS selected using different model selection strategies. b–d The average number of unbalanced baseline covariates before and after re-weighting on (b) train and test combined set, (c) train set, and (d) unseen test set. Three model selection strategies are (i) the AUC score on the validation fold during the cross-validation procedure, (ii) the cross-entropy loss on the validation fold, and (iii) our proposed strategy, which leverages balance performance on the training and validation combined folds and generalization performance on the validation fold. We reported drugs with ≥10% balanced trials among 100 emulations. A covariate is assumed balanced if its standardized mean difference (SMD) of its prevalence between exposure groups is at most 0.1 and a trial is assumed balanced if the ratio of unbalanced features among all covariates before/after re-weighting is ≤2%. The error bars indicate 95% confidence intervals by 1000-times bootstrapping. Welch’s t test (two-sample, two-sided) is used for testing the means of binary indicators for balanced trials, and p values and their associated significance marks are shown. *p < 0.05; **p < 0.01; ***p < 0.001; not significant with p ≥ 0.05 were not marked; AUC, area under the receiver operating characteristic curve. Source data are provided as a Source Data file.
