Introduction

Diarrhea is a leading cause of death globally, especially for children aged less than 5 years1. Approximately 25% of diarrheal episodes are likely attributed to bacterial pathogens2, which disproportionately contribute to severe outcomes including growth faltering and death3,4,5. Antibiotic therapy could prevent poor outcomes4,6,7, but in the absence of point-of-care diagnostics in most clinical facilities in low-resource settings, it is difficult to identify which episodes are bacterial and therefore could benefit from treatment. Currently, the World Health Organization (WHO) takes a syndromic approach by recommending antibiotics only for bloody diarrhea8. However, this approach misses watery bacterial diarrheal episodes that would likely be responsive to antibiotic therapy and inadvertently results in the widespread overuse of antibiotics because the guidance is perceived as insufficient in practice9. Strategies to better target antibiotic treatment are needed to optimize clinical outcomes and limit antibiotic resistance.

The factors that determine antibiotic treatment response may be more complex than bacterial etiology and antibiotic susceptibility. The AntiBiotics for Children with severe Diarrhea (ABCD) randomized clinical trial investigated the effects of azithromycin treatment among dehydrated or undernourished children with acute watery diarrhea, and the primary outcomes for the ABCD trial are published in ref. 10. The trial found no overall survival benefit, but a small improvement in linear growth among children who received azithromycin. Children with diarrhea attributed to bacterial etiologies were found to have reduced risk of diarrhea on day 3 and of hospitalization or death by day 906. Less expectedly, there was also reduced risk of these outcomes (albeit of smaller magnitude) among diarrheal episodes in which bacteria were detected but did not meet the quantity thresholds to be attributed to bacteria. Furthermore, bacterial etiology was not a strong determinant of the effect of azithromycin on linear growth outcomes; the effect of azithromycin was similar regardless of etiology. Together, these results suggest that the determinants of which children benefit from azithromycin treatment are complex and incompletely understood.

Other factors aside from diarrhea etiology have been related to azithromycin treatment benefit. The Macrolides Oraux pour Réduire les Décès avec un Oeil sur la Résistance (MORDOR) randomized trial identified heterogeneity in azithromycin effects on mortality by age and geographic setting11. The largest effects in MORDOR were observed in the lowest resourced setting (Niger) and among the youngest age group (1–5 months), suggesting child and sociodemographic characteristics may be important determinants of the benefit of azithromycin. However, the AVENIR trial found that all children 1–59 months should be treated to reduce mortality in the youngest infant age groups12. The biological mechanisms driving these differential treatment benefits are incompletely understood. Independent of diarrhea etiology, azithromycin is anti-inflammatory13, which may result in non-specific benefits for malnourished or otherwise vulnerable children. Relatedly, antibiotics are indicated without regard to specific pathogens in some cases; the WHO recommends routine treatment of children with severe acute malnutrition (SAM) with a broad-spectrum antibiotic14, and antibiotics have been considered for moderate malnutrition and environmental enteropathy10. Although heterogeneity in azithromycin treatment effects by individual-level characteristics has been observed in prior studies, it is unknown whether and to what extent these factors are informative for targeting treatment of diarrhea in the absence of, or in addition to, diagnostic testing.

In this secondary analysis of the ABCD trial, we used a machine learning-based framework to develop and evaluate personalized treatment rules for the decision to treat watery diarrhea with azithromycin. Through this process, we aim to interrogate whether certain types of data describing a child or their illness would be important for determining antibiotic treatment benefit. If data such as clinical or sociodemographic characteristics can sufficiently capture heterogeneity in the azithromycin treatment response, appropriate targeting of antibiotic treatment may be achievable even when point-of-care diagnostics are unavailable. By characterizing which children with watery diarrhea benefit from azithromycin and quantifying the expected benefits of personalized treatment rules for diarrhea, we identify innovative strategies to target azithromycin to those who benefit most and to limit antibiotic overuse and resistance.

Results

Among 6699 children in the ABCD trial who underwent quantitative polymerase chain reaction (qPCR) testing, we included 6692 (99.9%) children in this analysis. We excluded seven children due to missing or invalid results for bacterial pathogens to maintain consistency with a prior ABCD analysis6. The 1857 (27.7%) children that were missing the day 3 diarrhea outcome and 269 (4.0%) children that were missing the change in linear growth (length-for-age z-score [LAZ]) outcome were retained in the analysis via a nuisance regression that modeled the probability of having a non-missing outcome given azithromycin treatment assignment and all measured covariates (S. Methods). Therefore, we retained 6692 children in the analysis. Overall, 679 children had diarrhea on day 3, and 298 children were re-hospitalized or died by day 90, of which 26 (9%) children died.

Day 3 diarrhea outcome

For the day 3 diarrhea outcome, an average of 27.8% (n = 1860; 95% CI: 26.7%, 28.9%) of children were recommended for treatment under the personalized treatment rule based on all known diagnostic, child, and clinical characteristics (i.e., the comprehensive rule) using a threshold of a 7% reduction in risk of day 3 diarrhea (Fig. 1A and Table S4). Among those recommended for treatment, azithromycin was associated with an average 10.1% lower risk of diarrhea on day 3 (risk difference (RD), −0.101 [95% CI: −0.149, −0.054]), translating to a number needed to treat (NNT) of approximately 10 children to expect 1 fewer cases of diarrhea lasting until day 3 post-treatment. The effect of azithromycin was smaller among those for whom treatment was not recommended (average RD: −0.037 [95% CI: −0.063, −0.011]; NNT: 27). The etiologies of diarrhea were different between children recommended for treatment under the comprehensive rule compared to those not recommended for treatment (Table 1). Children recommended for treatment were more likely to have diarrhea attributed to bacteria. Child-level factors and characteristics of the clinical illness course were also different depending on the treatment recommendation. Children recommended for treatment were approximately 1 month older, were more malnourished (i.e., lower length-for-age, weight-for-age, and weight-for-length z-scores), had higher average numbers of loose and solid stools in the past 24 hours, had longer illness duration, and were less likely to be dehydrated or experience vomiting compared to those not recommended for treatment (Table 1). The relationships between child-level expected benefit and select clinical and child characteristics are also shown in Fig. S1.

Fig. 1: Comparison of average treatment benefit among those recommended for treatment and not recommended for treatment under each rule.
figure 1

A Comparison of average treatment benefit among those recommended and not recommended for treatment for diarrhea on day 3; B Comparison of average treatment benefit among those recommended and not recommended for treatment for re-hospitalization or death by day 90; C Comparison of average treatment benefit among those recommended and not recommended for treatment for change in linear growth. In all panels, the proportion recommended for treatment is denoted in parentheses after the label for each rule. Error bars represent a 95% confidence interval, and the sample size for each treatment benefit estimation is determined by the proportion of n = 6692 treated under each rule. CI Confidence interval, LAZ Length-for-age Z-score. Source data are provided as a Source Data File on Github.

Full size image
Table 1 Descriptive characteristics by treatment recommendation under the comprehensive rule with respect to each outcome (day 3 diarrhea, day 90 re-hospitalization or death, and change in length-for-age z-score)
Full size table

Overall, among rules based on subsets of clinical, diagnostic, and host characteristics, rules that incorporated information on bacterial pathogen quantities most closely approximated the benefit for day 3 diarrhea observed under the comprehensive rule. For example, the average benefit among children recommended treatment by the pathogen quantities rule was a 9.4% (95% CI: 14.0%, 4.7%) average absolute reduction in risk of day 3 diarrhea and by the pathogen + symptoms rule was a 9.5% (95% CI: 14.3%, 4.7%) average absolute risk reduction (Table S4). A similar proportion of children were treated by these rules and the comprehensive rule. While the Shigella rule recommended treatment for a smaller proportion of children (15.9% [n = 1112; 95% CI: 15.0%, 16.8%]), the average treatment benefit among those recommend treatment under the Shigella rule was larger (RD, −0.135 [95% CI: −0.202, −0.068]) than that under the comprehensive rule. Approximately 94% of children with Shigella-attributed diarrhea were treated, and the Shigella rule was more discriminating between children in that it produced the largest difference in average treatment benefits for day 3 diarrhea between those recommended for treatment and those not recommended for treatment (difference in RD: −0.095 [95% CI: −0.166, −0.024]) (Fig. 1A and Table S4). The overall average benefit among all children when treated or not treated according to the rules compared to no child treated were small and similar across rules (ranging from 1.2% to 2.6% absolute risk reduction), though the overall benefit of rules including bacterial pathogen quantities tended to be largest (Table S4). Compared to rules that did not include bacterial pathogen quantities, rules that included them generally had higher average sensitivity and specificity for identifying who should be recommended for treatment by the comprehensive rule (Tables 2 and S5). Finally, the child-level expected benefit under the pathogen quantities rule (concordance correlation coefficient (CCC): 0.76 [95% CI: 0.75, 0.77]) and pathogen + symptoms rule (CCC: 0.82 [95% CI: 0.81, 0.83]) were best correlated with the expected benefit under the comprehensive rule (Fig. 2A). The concordance of treatment assignment between rules in relation to the child-level expected benefit and select clinical and child characteristics is shown in Fig. S2.

Fig. 2: Concordant correlation coefficient between child-level expected benefits from the comprehensive and alternative treatment rules.
figure 2

A CCC of child-level expected benefits for diarrhea on day 3; B CCC of child-level expected benefits for re-hospitalization or death by day 90; C CCC of child-level expected benefits for change in linear growth. In all panels, the 95% confidence interval for the CCC is provided with sample size n = 6692. The dotted grey line represents perfect correlation (CCC = 1). CCC Concordant correlation coefficient, CI Confidence interval, LAZ Length-for-age Z-score. Source data are provided as a Source Data File on Github.

Full size image
Table 2 Sensitivity and specificity of each rule to identify children who were recommended treatment under the comprehensive rule
Full size table

Day 90 re-hospitalization or death outcome

For the day 90 re-hospitalization or death outcome, on average 38.0% (n = 2543; 95% CI: 36.9%, 39.2%) of children were recommended azithromycin treatment under the comprehensive rule with a threshold of a 2% absolute risk reduction (Table S4). Azithromycin was associated with a 2.4% lower risk of day 90 re-hospitalization or death (RD: −0.024 [95% CI: −0.041, −0.006]) among those for whom treatment was recommended, translating to a NNT of approximately 42 children to expect 1 fewer instance of re-hospitalization or death in the 90 days following treatment, compared to a 1.1% lower risk (RD: −0.011 [95% CI: −0.023, 0.000]; NNT: 91) among those for whom treatment was not recommended. The characteristics of children recommended for treatment based on the day 90 re-hospitalization or death outcome differed from those based on the day 3 diarrhea outcome (Table 1 and Fig. S1). Specifically, while children recommended for treatment were more likely to have a bacterial etiology of diarrhea, the differences in the prevalence of bacteria attributed diarrhea between children recommended and not recommended for treatment were smaller compared to those for the day 3 diarrhea outcome. In contrast, differences in markers of malnutrition between those recommended and not recommended treatment were more pronounced for day 90 re-hospitalization or death, and socioeconomic status (SES) was a strong determinant of the treatment recommendation. While similar proportions of children within each wealth quintile were recommended for treatment considering day 3 diarrhea, treatment was heavily skewed towards children in the lowest wealth quintiles when considering hospitalization or death (Table 1). Among children with moderate wasting or stunting, anthropometric measures were similar between children recommended for treatment versus those not, while differences in these measures between those recommended for treatment versus not were more pronounced among the subset of children with some or severe dehydration (Table S6).

The average treatment benefit for 90-day re-hospitalization or death among those recommended treatment was similar for all rules except the rule based on rotavirus only (RD: −0.003 [95% CI: −0.044, 0.038]) (Table S4). Similarly to the day 3 diarrhea outcome, the treatment benefit for day 90 re-hospitalization or death was greatest under the Shigella rule (RD, −0.030 [95% CI: −0.070, 0.010]), but the proportion treated was small (8.4% [n = 629; 95% CI: 7.7%, 9.2%]), and approximately 66% of children with Shigella-attributed diarrhea were treated. The overall average benefits among all children when treated or not treated according to the rules were largest for the host and host + symptoms rules (at approximately a 1% absolute risk reduction) and were null for the rules based on a single pathogen (Table S4).

Rules that included child-level characteristics (i.e., host rule, host + symptoms rule, comprehensive rule) were more discriminating between children with respect to treatment benefit than rules based on pathogens and/or symptoms in that they tended to produce larger differences in average treatment benefit between those recommended for treatment versus not recommended for treatment (difference in RD: −0.013 [95% CI: −0.035, 0.009], −0.015 [95% CI: −0.035, 0.006], −0.013 [95% CI: −0.034, 0.009], respectively) (Fig. 1B and Table S4). Although the treatment benefit difference between those recommended and not recommended for treatment was also larger for the Shigella rule (−0.014 [95% CI: −0.056, 0.027]), the estimate of the average treatment benefit among those recommended for treatment was imprecise. The average sensitivity of the host rules and host + symptoms rules to identify children recommended for treatment by the comprehensive rule were considerably higher than that for other rules (74.4% and 77.8%, respectively), though their specificities were lower than other rules (78.0% and 80.8%, respectively) (Tables 2 and S5). Generally, the child-level expected benefit from the host rule (CCC: 0.65 [95% CI: 0.63, 0.66]) and host + symptoms rule (CCC: 0.73 [95% CI: 0.72, 0.74]) were better correlated with the child-level expected benefit from the comprehensive rule than other rules (Fig. 2B), and treatment assignments from the host rule better aligned with those from the comprehensive rule (Fig. S2).

Linear growth outcome

For the change in linear growth outcome, an average of 37.6% (n = 2516; 95% CI: 36.5%, 38.9%) of children were recommended azithromycin treatment under the comprehensive rule using a threshold of 0.06 z-scores (Table S4). Azithromycin was expected to improve LAZ by 0.05 (95% CI: 0.01, 0.10) z-scores compared to the placebo among children recommended for treatment. The differences in the characteristics between children who were recommended and not recommended treatment according to the comprehensive rule for the linear growth outcome were less pronounced compared to those for the other outcomes (Table 1 and Fig. S1). Children recommended for treatment under the comprehensive rule were only slightly more likely to have diarrhea attributed to bacteria and had similar anthropometric measures compared to children not recommended for treatment. In contrast to rules for the other outcomes, children recommended treatment had shorter illness duration and were less likely to be dehydrated (Table 1).

None of the rules were particularly successful at distinguishing between children who did and did not benefit from treatment with respect to linear growth. Average changes in growth among those recommended treatment under the Shigella rule (0.08 z-scores [95% CI: 0.00, 0.16]), rotavirus rule (0.07 z-scores [95% CI: −0.01, 0.14]), and pathogen quantities rule (0.06 z-scores [95% CI: 0.01, 0.11]) were higher than that under the comprehensive rule (Table S4), and these rules had the largest differences in average benefit between those recommended for treatment versus those not (0.03 z-scores [95% CI: −0.05, 0.11], 0.02 z-scores [95% CI: −0.07, 0.10], 0.02 z-scores [95% CI: −0.04, 0.07], respectively), but all estimates were imprecise and not statistically different (Fig. 1C and Table S4). The remaining rules, including the comprehensive, resulted in little to no difference between the average change in LAZ among those recommended for treatment versus that among those not recommended treatment. Sensitivity for identifying children recommended treatment by the comprehensive rule was modest and highest for the host + symptoms rule (67.1%), host rule (55.0%), and pathogen + symptoms rule (54.4%) while specificity was highest for the Shigella rule (91.0%), rotavirus rule (90.3%), and host rule (81.7%) (Tables 2 and S5). There was no correlation between the expected benefit or treatment assignment between the comprehensive rule and other rules (Figs. 2C and S2).

Threshold comparison

We set thresholds at a 7% reduction in risk for day 3 diarrhea, 2% reduction in risk for day 90 re-hospitalization or death, and 0.06 difference in LAZ. In a sensitivity analysis, we examined how varying the threshold of clinical benefit that defines which children are recommended treatment impacted the proportion treated, expected benefits, and utility of different types of characteristics to define the rules. The proportion of children recommended for treatment by the comprehensive rule was dependent on the threshold of clinical benefit used to define which children were recommended treatment by the rule. For example, with a threshold of a 2% reduction in risk of diarrhea on day 3, the rule recommended approximately 85% of children for treatment versus recommending approximately 10% for treatment when the rule had a threshold of a 10% reduction in risk of diarrhea on day 3 (Fig. 3). This pattern, in which the proportion of children recommended for treatment decreased as the threshold of clinical benefit increased, was consistent for each of the outcomes. The average benefit among children recommended for treatment generally increased as the clinical benefit threshold increased for the day 3 diarrhea and day 90 re-hospitalization or death outcomes, though estimates at the largest thresholds lacked meaningful precision. In contrast, the average benefit in the linear growth outcome was similar across thresholds, with wide confidence intervals at the largest thresholds. Additionally, we found that the correlations between individual level expected benefit for (1) the pathogen quantities and comprehensive rules and (2) the host and comprehensive rules remained consistent over a range of thresholds (Fig. S3).

Fig. 3: Proportion treated and average benefit among those recommended for azithromycin under the comprehensive rule with varying thresholds of clinical benefit to define the rule.
figure 3

A Proportion treated and average benefit across thresholds for diarrhea on day 3; B Proportion treated and average benefit across thresholds for re-hospitalization or death by day 90; C Proportion treated and average benefit across thresholds for change in linear growth. The 95% confidence intervals are represented by the grey dotted lines with shaded bands, and the sample size for each treatment benefit estimation is determined by the proportion of n = 6692 treated under each threshold. RD Risk difference, CI Confidence interval, LAZ Length-for-age Z-score. Source data are provided as a Source Data File on Github.

Full size image

Application to external dataset

To demonstrate the generalizability and practical applicability of developing personalized treatment rules, we applied ABCD-learned rules for antibiotic treatment of day 3 diarrhea to the Global Enteric Multicenter Study (GEMS) (3), which was a seven site, case-control study for acute moderate-to-severe-diarrhea conducted between 2008 and 2011. Under the ABCD-learned comprehensive rule, the treatment effect across five seeds among children recommended for treatment in GEMS was −0.058 (95% CI: −0.189, 0.072) and among children not recommended for treatment was −0.008 (95% CI: −0.083, 0.067) (Table S7). The pathogen rule and host rule were also able to reasonably discern which children would benefit more from antibiotic treatment in GEMS. Similar to ABCD results, the child-level expected benefit under the pathogen rule applied to GEMS was correlated with that of the comprehensive rule (CCC: 0.69; 95% CI: 0.67, 0.71). Despite GEMS being non-randomized, including different study sites, and occurring six years prior, the rules learned in ABCD data were still able to identify heterogeneity in treatment effects against diarrhea duration in this external dataset.

Discussion

Although no overall survival benefit of azithromycin for moderately malnourished children with watery diarrhea was detected in the ABCD randomized trial10, our secondary analysis demonstrated that children can be targeted for treatment based on their diagnostic, malnutrition, and/or sociodemographic characteristics to improve outcomes and limit antibiotic use. Personalized treatment rules, such as those evaluated here, may strike a better balance between expected benefit and risks of antibiotic resistance than both current WHO guidelines, which miss children likely to benefit from antibiotics, and current practice, in which the majority of watery diarrhea episodes are treated with antibiotics8. While some of the rules included pathogen diagnostic data that are unlikely to be available at the point of care, other rules, such as the host rule or host + symptoms rule, only included information that would be easily observable in a clinical setting. While diagnostics were strong determinants of the effect of azithromycin on diarrhea duration, host characteristics were more relevant for identifying which children benefited most with respect to re-hospitalization or death. Therefore, appropriate targeting of antibiotics for children who would be expected to benefit in terms of the most severe outcomes may be achievable without widespread access to diagnostics. Furthermore, the rule could be adopted in clinical settings using a mobile based application, for example, without requiring the reasons for treatment recommendations to be completely interpretable. Deciding on the most appropriate personalized treatment rule in a given setting should weigh performance of the algorithm against feasibility with respect to data availability and limits on antibiotic use such as antibiotic resistance and common antibiotic side effects (i.e., nausea, abdominal pain, or vomiting). In the ABCD trial, serious events were limited to 1 participant in the azithromycin group experiencing seizure and 45 (1.1%) children in the azithromycin group requiring hospitalization or experiencing death by day 10, which was a similar proportion observed in the placebo group10. An example of deciding the most appropriate rule could be to increase the clinical benefit thresholds that define the rules, leading to fewer children being recommended an antibiotic, which may be preferred in settings with concerns about antibiotic resistance.

The finding that bacterial pathogen diagnostics were most informative for the azithromycin treatment response with respect to diarrhea duration is logical. Because antibiotics directly affect growth of bacteria that are the proximal cause of diarrhea symptoms, it is biologically plausible that pathogen quantities are the most useful information to predict improvements in short-term recovery from diarrheal symptoms. On the other hand, the results that targeting children for azithromycin based on host characteristics, particularly anthropometric measures, is more effective to improve day 90 re-hospitalization or death is consistent with WHO guidelines that recommend routine treatment of antibiotics for children with SAM14. Although the exact mechanism is not completely understood, hypotheses include that azithromycin reduces inflammatory responses and enteropathy that disrupt nutrient absorption, in addition to treating bacterial infections15. In one of our sensitivity analyses, we found that among a subset of children with stunting or moderate wasting, 57% of children were recommended for treatment by the comprehensive rule compared to 36% in the full study population, highlighting malnutrition as a driver of treatment decision for the day 90 re-hospitalization or death outcome (Table S6). For the change in LAZ outcome, we generally observed little to no heterogeneity in average treatment benefit across rules, suggesting that benefit for linear growth from azithromycin treatment is similar regardless of pathogen profile, symptoms, malnutrition indicators, and sociodemographic characteristics. Gut inflammation is one of several plausible reasons for impaired growth outcomes16, and the anti-inflammatory properties of azithromycin may similarly reduce inflammation and improve growth in all children. Given that mechanisms by which antibiotics affect diarrhea duration and re-hospitalization or death could be distinct, we would not expect a single treatment rule to identify the same children across different outcomes (Table S8).

The symptoms rule, which included information on days of diarrhea duration prior to enrollment, dehydration status, vomiting, and number of solid and loose stools in the prior 24 hours, did not recommend treatment for children similarly to the comprehensive rule for any outcome. This suggests that symptoms alone, information most proximal to the illness and readily available upon seeking care, may not be sufficiently informative for the decision to recommend antibiotics for children with acute watery diarrhea in these settings. Previous efforts to use clinical characteristics to inform treatment decisions have similarly found that epidemiologic factors like age, season, and anthropometry were most predictive of diarrhea etiology17,18.

The treatment rules we developed recommended treatment for children whose estimated expected benefit exceeded a threshold. However, in some instances when these rules were evaluated in the validation data, the average benefit of treatment was estimated to be smaller than this threshold. This is a feature of the cross-validation procedure used, which seeks to evaluate the generalization performance of the estimated treatment rules. Although these rules were designed with the intention of treating only children who would have an expected benefit larger than the threshold, the ultimate estimated benefit may vary due to random variability. We sought to minimize some of this variability by averaging results across five repeated tenfold cross-validation procedures.

This analysis was subject to limitations. First, the proportion recommended for treatment and estimates of average benefit were sensitive to the threshold chosen for treatment recommendation. For example, when we further increased the threshold for clinical benefit for the day 3 diarrhea outcome, a smaller proportion of children were treated, and the average treatment benefit among those recommended for treatment was stronger. However, the threshold for assigning treatment is inherently arbitrary and difficult to determine for outcomes like day 3 diarrhea, which are non-severe and may not justify treatment with antibiotics alone. Second, because we used machine learning algorithms to predict child-level expected benefits, we could not characterize exactly which and how individual variables contributed to each rule. Therefore, interpretation of associations between specific variables and treatment decisions was descriptive, and we do not report interpretable criteria for recommending treatment. Third, these results may not be generalizable to all children with watery diarrhea because the data were collected among vulnerable children accompanied by some or severe dehydration, moderate wasting, or severe stunting between 2 and 23 months of age. Children with SAM, who may be at highest risk for severe outcomes of diarrhea, were excluded. Enrollment was also limited to seven study sites in Southeast Asia and Africa. While we intentionally excluded site from the rules to ensure they could be applied across settings, residual effects of site may be reflected in other characteristics such as SES and malnutrition indicators. Fourth, mortality benefit from azithromycin treatment was not explicitly measured because it was a particularly rare outcome (n = 26 by day 90). Re-hospitalization and death by day 90 were combined as a single outcome to indicate severe outcomes while improving statistical power. Fifth, because the ABCD trial excluded children with bloody stool, we were unable to compare the performance of the rules with the WHO guidelines to evaluate their potential added value to the current recommendation. One important avenue for future research could involve comparing the performance of these treatment rules to the WHO guidelines in datasets where relevant data are available.

Using machine learning to compare alternative personalized treatment rules, we found that some diagnostic and host characteristics capture heterogeneity in azithromycin treatment response, and the most relevant characteristics differ by outcome. Our findings that pathogen quantities were important for treatment decisions related to day 3 diarrhea were expected. Interestingly, pathogen quantities were not necessary to identify which children to treat to prevent the most severe outcomes. Instead, measures of malnutrition and sociodemographic characteristics were more informative for the day 90 re-hospitalization or death outcome. These results highlight that diagnostics at the point of care in low resource settings may not be necessary to target treatment to prevent outcomes such as hospitalization or death but remain important for proximal outcomes such as illness duration. They also suggest antibiotic stewardship is feasible by demonstrating that antibiotics can be limited to the children who will benefit the most.

Methods

For the ABCD trial, ethical review committees from the WHO and each respective country institution approved the trial protocol, and informed consent was obtained from all caregivers of included participants. This study involving secondary analysis of the trial data obtained non-humans subject research determination from Emory University Institutional Review Board.

Study design

The ABCD trial (NCT03130114) enrolled 8268 dehydrated or undernourished children with acute watery diarrhea across seven sites in Bangladesh, India, Kenya, Malawi, Mali, Pakistan, and Tanzania between 2017 and 2019. Study procedures and methods for the ABCD trial have been described in prior publications6,10. Briefly, children 2–23 months presenting to healthcare with acute watery diarrhea were eligible if they had some or severe dehydration, moderate wasting, or severe stunting and were excluded if they had dysentery, suspected cholera, SAM, or other infections requiring antibiotics. At enrollment, research staff collected clinical history (vomiting, number of loose stools in prior 24 hours, number of solid stools in prior 24 hours, number of days of diarrheal illness, dehydration status), measures of malnutrition (average middle-upper arm circumference, weight-for-age z-score, LAZ, and weight-for-length z-score), relevant demographic information (age, sex, socioeconomic status categorized in quintiles (based on country-specific wealth distributions in the Demographic and Health Survey10), number of children <5 in household), and fecal samples from all participants. Biospecimens, including whole stools and rectal swabs, for the first approximately 1000 children enrolled at each site underwent pathogen testing by qPCR using the TaqMan Array Card platform. Details of the laboratory methods and quality control have been previously described6. Our analysis considers both the presence and relative quantity (based on log-10 transformed qPCR cycle threshold values) of the 12 most-detected pathogens in the study: rotavirus, norovirus, adenovirus, astrovirus, sapovirus, ST ETEC, Shigella, Campylobacter, tEPEC, V. Cholerae, Salmonella, and Cryptosporidium.

After enrollment, children were randomized 1:1 to receive a 3-day course of azithromycin (10 mg/kg) or placebo and were subsequently followed for up to 180 days. Follow-up visits occurred on days 2, 3, 45, 90, and 180 post-enrollment, during which vital status, anthropometric, clinical, and autopsy data were collected when relevant. This analysis focuses on 3 outcomes: (1) presence of diarrhea on day 3 following enrollment, (2) hospitalization following the enrollment-associated episode or death by day 90 following enrollment, and (3) change in LAZ between days 0 and 90. Presence of diarrhea on day 3 and hospitalization or death by day 90 were reported by the child’s caregiver at follow-up10. For LAZ, length and weight were measured according to WHO standard procedures19, and LAZ was calculated using WHO Child Growth Standards20.

Treatment rules

We developed personalized treatment rules based on all available child and clinical characteristics as well as seven pre-specified sets of covariates based on the type of data and considering plausible scenarios of data availability and/or feasibility of implementation in real-world settings (Table 3). Comparison of results across rules allowed us to interrogate the relevance of different types of data as determinants of the heterogeneity in the treatment effects. The rule based on all available child, clinical, and diagnostic characteristics was considered the comprehensive rule since it included the most data compared to other rules.

Table 3 Description and rationale of personalized treatment rules
Full size table

Treatment rules were developed using a robust ensemble machine learning-based procedure to estimate the expected benefit of azithromycin (i.e., additive risk reduction) to a child with respect to a particular outcome (e.g., presence of diarrhea on day 3 following enrollment) given a set of one or more characteristics of the child. Rules for treating children were defined based on this child-level expected benefit, whereby all children whose expected benefit exceeded a certain clinically relevant threshold were recommended azithromycin by the rule. The machine learning-based procedure combines predictions from a library of user-specified candidate regression models into an optimally weighted ensemble model. For additional details on the machine learning pipeline for learning and evaluating treatment rules, see Supplementary Materials Methods.

Statistical analysis

The study cohort was characterized by providing summary statistics of demographic and clinical characteristics. The primary analysis involved a two-step process involving randomly subsetting data into a training and validation sample. In the first step, we used the training sample to learn the child-level expected benefit for treatment given a set of input characteristics and developed a rule for treating children based on the level of expected benefit. In the second step, the validation sample was used to identify the children that would be recommended treatment given their expected benefit and to calculate the diagnostic performance of the rule.

For the day 3 diarrhea outcome, we focus our presentation on treatment rules that treat all children who have an expected benefit of at least 7% absolute reduction in risk of day 3 diarrhea, while treating no children with expected benefit <7% risk reduction. For the day 90 re-hospitalization or death outcome, we focus on treatment rules that treat children who have an expected 2% or greater absolute reduction in risk of hospitalization or death. For the change in LAZ outcome, we focus on rules that treat children who have an expected improvement of at least 0.06 units of LAZ. These outcome-specific cut-offs were based on a clinically relevant expected benefit and such that the proportion of children recommended treatment (30–45%) was likely to be acceptable considering concerns of antibiotic overuse and risks of antibiotic resistance. In sensitivity analyses, we varied these thresholds.

We quantified the impact of each treatment rule by estimating: (i) the proportion of children treated under the rule; (ii) the average benefit of treatment among those recommended treatment under the rule; (iii) the average benefit of treatment among those not recommended treatment under the rule; and (iv) the average benefit among all children of being treated according to the rule. While (ii) and (iii) compare the impact of azithromycin versus placebo among a subset of children, (iv) compares among all children the impact of treating all children according to the treatment rule (whereby some children are treated with azithromycin and others with placebo) versus treating all children with placebo. Estimate (iv) reflects comparing a scenario where the treatment algorithm was adopted for all children with watery diarrhea to a scenario where no child was treated. We then evaluate how well the rules discriminate between children in terms of their expected benefit by estimating the difference between the average benefit among those recommended for treatment (ii) with that among those not recommended for treatment (iii).

To avoid overfitting and to obtain an unbiased evaluation of the performance of our treatment rules, we utilized tenfold cross-validation wherein data were randomly divided into ten distinct partitions, treatment rules were learned in a training sample and evaluated in a held-out validation sample. Cross-validated evaluation of the performance of treatment rules likely provides reasonable inference and approximation to the estimated performance of the true optimal treatment rule given a particular set of values21,22,23. To reduce the randomness inherent in this sample splitting procedure24, we then repeated the entire process five times and present point estimates and 95% confidence intervals for the average performance of the rules over the five repeated cross-validation procedures.

To better understand which types of data were strong determinants of the machine learned treatment rules, we present summary measures of child-level characteristics stratified by the decision to recommend treatment under the comprehensive rule. Because results were repeated across five seeds, we presented the distribution of characteristics based on treatment recommendation from the seed with the median expected child-level benefit for the descriptive analyses. We also evaluate how well the rules approximate the comprehensive rule by estimating the sensitivity and specificity of each rule to identify children recommended for treatment by the comprehensive rule and the CCC, which measures the level of agreement of child-level expected benefit, a continuous measure of the risk difference, obtained by two different treatment rules, the comprehensive and another rule. We expect that rules, that group variables into categories (Table 3), that best approximate the comprehensive rule signal which variables may be important for determining treatment benefit.

To further interrogate the role of the threshold for our results, we estimated the CCC between individual level expected benefit for (1) the pathogen quantities and comprehensive rules and (2) the host and comprehensive rules across a range of thresholds. We assessed the reactivity of the correlation between treatment rules to the threshold using the day 90 re-hospitalization and death outcome, which would be most relevant for treatment guideline changes.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.