Introduction

Dyskalemia refers to abnormalities in serum potassium concentration in the bloodstream, which is typically detected from the laboratory potassium concentration (Lab-K+); a common and life-threatening physiological condition, dyskalemia encompasses both hypokalemia and hyperkalemia1,2. Dyskalemia is frequently observed in patients in the emergency department (ED) and general hospital wards, with an increased prevalence observed in patients with advanced chronic kidney disease (CKD) or acute kidney injury3,4,5,6,7, among whom ~20% present with hypokalemia and 2–5% with hyperkalemia. Our recent study also revealed that 3% and 1% of all patients visiting the ED have moderate-to-severe hypokalemia [serum potassium (Lab-K+) ≤ 3.0 mmol/L] and hyperkalemia (Lab-K+ ≥6.0 mmol/L), respectively8. Owing to its detrimental effects on the incidence of arrhythmia and sudden cardiac death, dyskalemia is associated with significant morbidity and mortality9. The U-shaped association between dyskalemia and mortality has been well studied in patients with diseases such as diabetes mellitus, CKD, acute myocardial infarction, and heart failure10. Since dyskalemia is potentially fatal, timely recognition with prompt treatment is essential. However, the diagnosis of dyskalemia relies on blood tests with varying turnaround times11, often resulting in delayed recognition and treatment12,13. To date, therefore, the management of dyskalemia remains a great clinical challenge.

The effect of dyskalemia on the cardiovascular system presents as significant abnormalities on electrocardiography (ECG)14. However, the sensitivities of first-line physicians using ECG in identifying hyperkalemia and hypokalemia are relatively low, ranging between 18 and 21% and between 17 and 51%, respectively7,15,16. Despite the typical hyperkalemia features observed on ECG, early treatment is not initiated13. With the revolution of deep learning, artificial intelligence (AI)-based ECG systems (AI-ECGs) have been validated in identifying dyskalemia with much greater accuracy than medical experts7,8,17,18. AI-enabled dyskalemia using ECG (AIDE) has demonstrated excellent performance in detecting moderate-to-severe hyperkalemia [area under the receiver operating characteristic (ROC) curve (AUC) > 0.955] and hypokalemia (AUC > 0.858) in the ED8. Moreover, we have also reported that patients with normal Lab-K+ but ECG-dyskalemia (pseudopositive) also exhibit more comorbidities and worse cardiovascular outcomes8.

Recently, we published a randomized controlled trial (RCT) that demonstrated how AI-ECG alerts can reduce mortality19. However, the precise mechanisms underlying how the implementation of AI-ECG alerts leads to decreased mortality remain to be elucidated. The AIDE alert is highly specific and can aid in the development of concrete treatment strategies for managing potassium (K+) imbalance, but its effectiveness has not yet been validated. In this multisite single-blind physician-level RCT, we aimed to evaluate the effect of AIDE alerts, providing real-time ECG analysis for recommending dyskalemia treatment, by integrating an AI-ECG into the electronic health records (EHRs) accessible to clinical physicians in the ED. Our primary objective was to assess whether the AIDE alert facilitated prompt treatment decisions in the intervention group compared with the control group, where treatment initiation is dependent on serum laboratory results or traditional ECG interpretation.

Results

Patient characteristics stratified by randomized group and AIDE use

A total of 70 physicians participated in this trial over the 6-month period in 2022 (Fig. 1). Their characteristics are shown in Supplemental Table 1, with 35 assigned to the intervention group (mean age, 34.3 years; 29 biological males) and 35 to the control group (mean age, 32.9 years; 30 biological males). During the study period, 7638 and 7651 ED patients received at least one ECG examination in the intervention and control groups, respectively. Patients who were under 18 years old or who received dyskalemia-related treatment before undergoing the ECG examination were excluded. All the ECG scans were processed by the AIDE system; however, only the physicians in the intervention group received a real-time pop-up alert, while AIDE data from the control group were hidden but nevertheless collected for post hoc analysis. As shown in Table 1, a total of 7506 and 7483 patients were included in the intervention and control groups, respectively. The AIDE system identified 81 patients (1.1%) with ECG-detected moderate-to-severe hyperkalemia and 172 patients (2.3%) with ECG-detected moderate-to-severe hypokalemia in the intervention group; compared with 77 patients (1.0%) with ECG-detected moderate-to-severe hyperkalemia and 192 patients (2.6%) with ECG-detected moderate-to-severe hypokalemia in the control group. The mean age was 62.2 ± 19.6/62.1 ± 19.5 years, and 49.8%/50.5% of the participants in the intervention/control groups, respectively, were biological males. There was no significant difference following multiple comparison correction between the intervention and control groups, indicating good comparability.

Fig. 1: CONSORT-AI flow diagram.
Fig. 1: CONSORT-AI flow diagram.
Full size image

AIDE alert context: “An ECG from patient X indicates a high risk for hyperkalemia/hypokalemia. The estimated ECG-K+ is X. Please assess patient data to arrange suitable treatment. According to the previous study, you may also need to assess the risk of cardiac events. Please click on the following link to connect the ECG and the result of AI-ECG prediction.” ED emergency department, ECG electrocardiography, AIDE artificial intelligence-enabled dyskalemia using electrocardiogram.

Table 1 The critical information and patient characteristics stratified by randomization and AIDE
Full size table

Compared with patients in the ECG-normal and hypokalemia subgroups, patients in the ECG-hyperkalemia subgroup were older and had significantly more comorbidities, such as CKD, diabetes mellitus, hypertension, hyperlipidemia, coronary artery disease and atrial fibrillation. Compared with patients in the ECG-normal and ECG-hyperkalemia subgroups, patients in the ECG-hypokalemia subgroup exhibited more severe medical conditions. On the basis of the available Lab-K+ data, the prevalence of moderate to severe Lab-hyperkalemia and Lab-hypokalemia was only 0.3–0.4% and 2.2%, respectively. Underestimation was likely because many patients at risk of potential dyskalemia were treated before blood sampling, as shown by the greater Lab-K+ sampling after treatment in the ECG-hyperkalemia (48.1%) and ECG-hypokalemia (19.5%) groups than in the ECG-normal (14.8%) group. Follow-up Lab-K+ data for patients receiving dyskalemia-related treatment are shown in Supplemental Table 2.

Post hoc analysis for assessing the prospective accuracy of AIDE in detecting dyskalemia

We used paired ECG and Lab-K+ data within one hour before treatment to evaluate the accuracy of AIDE. AIDE achieved AUCs (Fig. 2a) of 0.971 and 0.906 in detecting moderate-to-severe hyperkalemia (Lab-K+ ≥6.0 mmol/L) and hypokalemia (Lab-K+ ≤3.0 mmol/L), with satisfactory areas under the precision recall curves (PRAUCs) of 0.531 and 0.347 (Fig. 2b), respectively. The prospective positive predictive value for both ECG hyperkalemia (43.6%) and hypokalemia (42.7%) in this RCT was greater than anticipated (40%). Lab-K+ values for severe hyperkalemia (Lab-K+ ≥6.5 mmol/L)/hypokalemia (Lab-K+ ≤2.5 mmol/L) and satisfactory AUCs of 0.964/0.987 and PRAUCs of 0.592/0.407 were also evident (Supplemental Fig. 1). As shown in Fig. 2c, the r and MAE values were 0.60 and 0.36, respectively, between Lab-K+ and ECG-K+. The mean difference between Lab-K+ and ECG-K+ was −0.03 (95% interval: −0.97 to 0.90) (Fig. 2d). In Supplemental Fig. 2, we present the algorithm performance for different K+ ranges (e.g., hyperkalemia versus hypokalemia).

Fig. 2: Prospective accuracy of AIDE for estimating blood potassium.
Fig. 2: Prospective accuracy of AIDE for estimating blood potassium.
Full size image

This analysis included only ECGs with an associated Lab-K+ collection within 1 hour of the ECG examination (n = 11,547; 5781 in the intervention group and 5766 in the control group), regardless of whether the blood sample was drawn before or after the ECG. Data from patients whose Lab-K+ measurements were obtained >1 hour apart or after treatment were excluded. We combined the intervention and control groups in this analysis. The ROC curve and PRROC curve were used to determine the performance of AIDE for hyperkalemia (Lab-K+ ≥6.0 mmol/L, a and hypokalemia (Lab-K+ ≤3.0 mmol/L, b), respectively. The cutoff points were determined as previously expected to have a positive predictive value of 40%. Scatter plots (c) and Bland–Altman plots (d) were used to compare the Lab-K+ and ECG-K+ values. Red points represent the highest density, followed by yellow, green, light blue, and dark blue. AUC area under the curve, Diff mean difference, r Pearson correlation, MAE mean absolute error, Sens. sensitivity, Spec. specificity, PPV positive predictive value, NPV negative predictive value, PRAUC precision recall curve. Source data are provided as a Source Data file.

Prespecified primary analysis: hyperkalemia-related treatment

Overall, the rate of hyperkalemia-related treatment was not significantly greater in the intervention group (8.0%) than in the control group (7.7%) (HR 1.05 and 95% CI 0.94–1.17; p = 0.420; Fig. 3a). This outcome was consistent with expectations before the trial began, since the true impact was anticipated to be found in the small subset of patients with ECG-detected hyperkalemia. Therefore, our prespecified primary analysis also included a subgroup analysis focusing on patients with ECG-detected hyperkalemia. As shown in Fig. 3b, a significantly greater proportion of the intervention group underwent hyperkalemia-related treatment within three hours (69.1% in the intervention group versus 41.6% in the control group, HR 2.23 and 95% CI 1.44–3.46, p < 0.001) in the AI-identified ECG-hyperkalemia subgroup, while the non-ECG-hyperkalemia subgroup showed no significant difference. Notably, this significant difference in hyperkalemia-related treatment was achieved within 1 hour (42.0% in the intervention group versus 15.6% in the control group; HR 3.13; 95% CI 1.58–6.20; p < 0.001).

Fig. 3: Prespecified primary analysis.
Fig. 3: Prespecified primary analysis.
Full size image

All the significance tests were based on the Cox proportional hazard mixed effect model. a The overall effect of AIDE intervention on hyperkalemia-related treatment, including drugs (calcium chloride, insulin, potassium binding resin, β2-agonist, loop diuretics, or sodium bicarbonate) and hemodialysis, within 3 hours. b Effect of AI-ECG intervention stratified by AIDE prediction (ECG-hyperkalemia and non-ECG-hyperkalemia). For patients identified as having ECG-hyperkalemia in the intervention group, the corresponding physician received a “pop-up” alert in the hospital information system. Since the prespecified treatments for ECG-hypokalemia differ from those for ECG-hyperkalemia, we combined ECG-hypokalemia and ECG-normal into ECG-non-hyperkalemia in this analysis. We highlighted the difference within 1 and 2 hours. c The overall effect of AIDE intervention on hypokalemia-related treatment, including intravenous potassium chloride, oral potassium gluconate, and oral potassium chloride, within 3 hours. d The effect of AI-ECG intervention stratified by AIDE prediction (ECG-hypokalemia and non-ECG-hypokalemia). For patients identified as having ECG-hypokalemia in the intervention group, the corresponding physician received a “pop-up” alert in the hospital information system. Since the prespecified treatments for ECG-hyperkalemia differ from those for ECG-hypokalemia, we combined ECG-hyperkalemia and ECG-normal into ECG-non-hypokalemia in this analysis. Cox proportional hazard mixed effect models without covariates were used for the statistical test, which was two-sided, with no adjustment for multiple comparisons. The blue and brown lines represent the intervention and control groups, respectively. The tables show the at-risk population and cumulative risk for the given time intervals in each group. Source data are provided as a Source Data file.

Prespecified primary analysis: hypokalemia-related treatment

As shown in Fig. 3c, the rate of hypokalemia-related treatment was not significantly greater in the intervention group (2.1%) than in the control group (2.4%) (HR 0.91 and 95% CI 0.74–1.13, p = 0.392). Stratified analysis also revealed no significant effects of AIDE in the ECG-hypokalemia subgroup or non-ECG-hypokalemia subgroup (Fig. 3d). With respect to ECG hypokalemia alerts, no additional hypokalemia-related treatments were administered within three hours (HR 1.14, 95% CI 0.77–1.77; p = 0.551) in the intervention group (31.4%) compared with the control group (29.7%).

Prespecified safety outcomes and secondary outcomes

Comparisons of safety and clinical outcomes between the intervention and control groups are shown in Fig. 4a. In the intervention and control groups, there were 27 versus 32 treatment-induced hypokalemia events (HR 0.84; 95% CI 0.50–1.40; p = 0.507), 0 versus 0 cases of treatment-induced hyperkalemia, 49 versus 44 cases of cardiac arrest (HR 1.11; 95% CI 0.74–1.67; p = 0.608), 692 versus 631 incidents of all-cause mortality (HR 1.11; 95% CI 1.00–1.24; p = 0.053), 14 versus 12 incidents of electric shock (HR 1.16; 95% CI 0.54–2.52; p = 0.701), 12 versus 6 incidents of cardiopulmonary resuscitation (CPR, HR 2.00; 95% CI 0.75–5.32; p = 0.167), and 6491 versus 6572 patients discharged within 14 days (HR 0.97; 95% CI 0.94–1.01; p = 0.126), respectively.

Fig. 4: Safety outcomes and clinical outcomes stratified by AIDE predictions.
Fig. 4: Safety outcomes and clinical outcomes stratified by AIDE predictions.
Full size image

The total sample size consisted of 14,989 patients (7506 in the intervention group and 7483 in the control group). Within this cohort, the subgroup sizes were as follows: ECG-hyperkalemia (n = 158), ECG-hypokalemia (n = 364), and ECG-normal (n = 14,467). The error bars denote 95% confidence intervals. All the significance tests were based on the Cox proportional hazard mixed effect model. Each p value was two-sided, with no adjustment for multiple comparisons. a Results for the overall population. b Prespecified subgroup analyses of AI-ECG prediction. The p value for interactions was two-sided, with no adjustment for multiple comparisons. For treatment-induced hypokalemia, we included only events after hyperkalemia-related treatment (calcium chloride, insulin, potassium-binding resin, β2-agonist, loop diuretics, sodium bicarbonate, or hemodialysis). For treatment-induced hyperkalemia, we included only events that occurred after hypokalemia-related treatment (intravenous potassium chloride, oral potassium gluconate, and oral potassium chloride). In the analysis for discharge within 14 days, patients who were not hospitalized and who directly left the ED were considered to be discharged within 0 days. A hazard ratio (HR) > 1 was better for the intervention group than for the control group for discharge within 14 days, and an HR < 1 was better for the intervention group than for the control group for the other outcomes. Source data are provided as a Source Data file.

The prespecified subgroup analyses based on AI-ECG predictions are shown in Fig. 4b. Although the difference was not statistically significant (2 versus 1, HR 1.93, 95% CI 0.17–21.23, p = 0.593), in the ECG-hyperkalemia subgroup, we paid particular attention to treatment-induced hypokalemia because it represents a side effect of the most important finding from the primary analysis—faster hyperkalemia-related treatment. Two patients had transient treatment-induced hypokalemia after emergent hemodialysis using low K+ dialysate (1.0 mmol/L) for severe hyperkalemia with a Lab-K+ concentration higher than 7.0 mmol/L in the ECG hyperkalemia intervention group, and the Lab-K+ concentration quickly returned to normal within a few hours. One patient in the control group with acute critical illness required intubation, inotropic agent administration, and aggressive sodium bicarbonate administration to maintain vital signs before the laboratory report was obtained, with development of hypokalemia upon stabilization. There were no significant findings on clinical outcomes in the ECG-hypokalemia, ECG-normal, or ECG-hyperkalemia subgroups.

Prespecified exploratory outcomes

Overall, the impact of AIDE for hyperkalemia on increased hyperkalemia-related treatment was largely consistent among different comorbid diseases (Supplemental Fig. 3). The impact of AIDE on hyperkalemia under different Lab-K+ conditions is shown in Supplemental Fig. 4. Specifically, patients were categorized into three main groups on the basis of whether (1) Lab-K+ was ordered after the ECG or if it was collected after treatment, (2) the Lab-K+ test and ECG were ordered simultaneously, and (3) the Lab-K+ report was available before ECG was performed. In group 3 (n = 11 and 20 in the intervention and control groups, respectively), we further identified patients with (Lab-K+ ≥ 6.0 mmol/L) and without hyperkalemia (Lab-K+ < 6.0 mmol/L), as well as whether the laboratory test was performed within or more than one hour prior to the ECG. These analyses aimed to explore the impact of nonhyperkalemia laboratory reports on interventions when an AIDE alert was issued. However, all the results were nonsignificant, likely because of the small sample size. For the ECG-hypokalemia subgroup, a detailed stratified analysis by patient characteristics is shown in Supplemental Fig. 5, and all the results were nonsignificant and largely consistent.

Other post hoc analyses

The medications used for hyperkalemia-related treatment for each subgroup are shown in Supplemental Fig. 6a. In the ECG-hyperkalemia subgroup, the most common medications for initial treatment included intravenous calcium chloride (66.1%), followed by insulin, loop diuretics, and sodium bicarbonate in the intervention group, whereas intravenous calcium chloride (84.4%) and insulin administration were first initiated in the control group. As shown in Supplemental Fig. 6b, there were also significant increases in the combined use of calcium chloride, insulin, and sodium bicarbonate within three hours of hyperkalemia-related treatment.

The results of the severity stratification analysis for ECG-based hyperkalemia on hyperkalemia-related treatment are shown in Supplemental Fig. 7. AI-ECG alerts were more effective in the intervention group for both the ECG-moderate-hyperkalemia subgroup within three hours (62.1% in the intervention group versus 33.3% in the control group; HR 2.43 and 95% CI 1.39–4.23; p = 0.002) and the ECG-severe-hyperkalemia subgroup (82.6% in the intervention group versus 62.5% in the control group; HR 2.01 and 95% CI 1.01–4.01; p = 0.047). The continuous association between the severity of ECG-related hyperkalemia and clinical action suggests that severity was the major determining factor of whether to initiate dyskalemia-related treatment in the clinical setting. Conversely, the AI-ECG assisted physicians in identifying ECG-moderate hyperkalemia. Supplemental Fig. 8 shows the efficacy of hypokalemia-related treatment stratified by the severity of ECG-related hypokalemia. Interestingly, ECG hypokalemia alerts did not significantly affect treatment, even in the severe ECG-related hypokalemia subgroup.

A comprehensive review of patients on hyperkalemia-related treatment in the ECG-related hyperkalemia subgroup was performed (Supplemental Table 3), which revealed that significantly more hyperkalemia-related treatments (p = 0.357) were administered before Lab-K+ in the intervention group (55.4%) than in the control group (43.7%). We chose a Lab-K+ treatment threshold of 5.5 mmol/L rather than 6.0 mmol/L to align with standard clinical practice, where treatment is often initiated at this level to prevent further deterioration and cardiac arrhythmia. Most patients met the treatment criteria (Lab-K+ value ≥ 5.5 mmol/L) in the intervention (64.2%) and control (68.8%) groups. For those without laboratory evidence of hyperkalemia—those who may only have had a posttreatment Lab-K+ value < 5.5 mmol/L within three hours—we found that most had a recorded history of hyperkalemia or subsequently demonstrated hyperkalemia during hospitalization (intervention group: 19.6%; control group: 18.7%; p = 0.940). With respect to the remaining patients without Lab-K+ validated hyperkalemia (≥5.5 mmol/L) in the post hoc medical records review, 3 of 12 patients (25%) in the intervention group presented with Lab-K+ values ranging from 5.0–5.4 mmol/L after treatment. The numbers of patients with Lab-K+ <4.5 mmol/L after treatment in the intervention (n = 3) and control (n = 3) groups were similar (p = 0.448). Among patients who did not receive hyperkalemia-related treatment within three hours in the ECG-hyperkalemia subgroup (Supplemental Table 4), 14 (56.0%) and 32 (72.7%) patients in the intervention and control groups, respectively, still received hyperkalemia-related treatment prior to discharge (p = 0.250). Nine (36.0%) and 26 (57.8%) patients met the treatment criteria (Lab-K+ concentration ≥ 5.5 mmol/L) in the intervention and control groups (p = 0.194), respectively.

We explored the interplay between AIDE alerts, laboratory testing, and subsequent hyperkalemia treatment decisions. Supplemental Fig. 9a shows that physicians in the intervention group appeared more likely to order lab tests after receiving an AIDE alert, suggesting that the alert itself might prompt additional confirmation of dyskalemia (62.3% in the intervention group versus 43.0% in the control group; HR 1.95; 95% CI 1.09–3.48; p = 0.024). Furthermore, when the treatments before and after lab report issuance were analyzed separately (Supplemental Fig. 9b), the main difference between the intervention and control arms arose from treatments given prior to the lab report (38.3% in the intervention group versus 18.2% in the control group, HR 2.31 and 95% CI 1.23–4.34, p = 0.010). After the laboratory results became available, the incremental difference in treatment rates between the two groups was relatively small (30.9% in the intervention group versus 23.4% in the control group; HR 1.43 and 95% CI 0.78–2.63; p = 0.244), implying that the AI-ECG alert influenced critical clinical decisions made during the early phase of patient care.

We explored whether the effect of the AIDE alert varied by physician demographics. The hyperkalemia-related treatment rates stratified by physician age, training level, professional background, and sex are shown in Supplemental Fig. 10. We observed that younger physicians (<35 years old) and resident doctors appeared more likely to address the AIDE alert, although many of these subgroup differences did not reach statistical significance.

We investigated whether the recommendation embedded in the AIDE alert—stating that pseudopositive ECG dyskalemia may pose a higher cardiac risk—prompted additional diagnostic evaluations. The data in Supplemental Fig. 11 demonstrate that physicians in the intervention group were more likely to order troponin I (HR: 1.86; 95% CI: 1.10–3.17; p = 0.022), NT-proBNP (HR: 2.99; 95% CI: 1.26–7.07; p = 0.013), or D-dimer (HR: 2.30; 95% CI: 1.21–4.36; p = 0.011) tests for the ECG-hyperkalemia subgroup and echocardiogram (HR: 1.68; 95% CI: 1.07–2.64; p = 0.024) for the ECG-hypokalemia subgroup. Supplemental Fig. 12 further stratifies these findings by the ECG-hyperkalemia and ECG-hypokalemia subgroups, suggesting that the alert’s emphasis on potential cardiac risk led to a modest increase in newly discovered cardiac abnormalities in the intervention arm.

Discussion

In the current literature, very few AI algorithms for medical applications have been prospectively evaluated in RCTs20. The benefit of timely correction of dyskalemia is widely recognized21,22; therefore, in this study, we examined the incorporation of AIDE into EHR for guiding clinical decision-making among ED physicians. This prospective study confirmed the satisfactory performance of AI-ECG in detecting moderate-to-severe dyskalemia and revealed that AIDE led to significantly earlier intervention for ECG-related hyperkalemia. Notably, the risk of dyskalemia overcorrection was minimal. However, the evidence demonstrating the benefit of AIDE alerts on clinical outcomes was not significant.

Dyskalemia can lead to significant morbidity and mortality if not recognized and managed in a timely manner1. Owing to the increased cardiac arrhythmia and death associated with dyskalemia, the Kidney Disease: Improving Global Outcomes (KDIGO) Controversies Conference recommends severity assessment and associated ECG changes for the management of dyskalemia23. Since a higher mortality rate is observed in patients with dyskalemia accompanied by ECG abnormalities24, identifying this population is crucial25,26. Following these recommendations, our trial implements AIDE to detect moderate to severe dyskalemia and early management. The prospective accuracy of AIDE for hyperkalemia (AUC = 0.971) and hypokalemia (AUC = 0.906) detection was even better than that reported in our previous retrospective cohort study (AUC for hyperkalemia >0.955; AUC for hypokalemia >0.858)8, revealing an improvement in sensitivity of 17–43%7,15,16,27. The AIDE system exhibited increased scatter at higher and lower K+ levels and rarely predicted values > 7.0 mmol/L or <2.0 mmol/L for patients with laboratory-confirmed K+ levels ranging from 3.0 to 6.0 mmol/L. This interventional study revealed that dyskalemia was managed early despite the lack of available Lab-K+ for accurate analysis. Therefore, the prevalence of dyskalemia in the study population might be lower than the expected value, leading to an underestimation of the positive predictive value. Nevertheless, the positive predictive values (43.6% and 42.7%) of patients with dyskalemia were still higher than previously anticipated (40%)7,8,17,18, highlighting the clinical potential of AIDE in the real-world setting.

Clinical decision support systems (CDSSs) significantly improve the management of diseases such as heart failure with reduced ejection fraction28 and acute kidney injury29. With the advancement of AI technology, AI-embedded CDSSs may assist in improving the precision and efficacy of clinical practices; however, validation and evidence are currently lacking. In the present study, which utilized AIDE as a CDSS, the use of medication to treat hyperkalemia was significantly greater in the intervention group than in the control group. AIDE intervention also revealed that 42.0% of patients received hyperkalemia-related treatment within one hour, which was usually earlier than when Lab-K+ became available. With respect to the severity of hyperkalemia, prominent ECG changes associated with severe hyperkalemia made the diagnosis more apparent for frontline physicians without AIDE (ECG-severe hyperkalemia). We found that 62.5% of the patients with ECG-severe hyperkalemia received immediate hyperkalemia-related treatment, even in the control group, compared with 33.3% of patients with ECG-moderate hyperkalemia. With the AIDE alert, a significant increase in hyperkalemia-related treatment was further observed in the intervention group (82.6% for ECG-severe hyperkalemia and 62.1% for ECG-moderate hyperkalemia). We assume that most physicians recognize the urgency of ECG changes associated with severe hyperkalemia; however, prompt intervention may not always be ordered in the clinical setting because of limited experience and the inability to accurately diagnose the condition. Given that the sensitivities for frontline ED physicians to recognize ECG changes associated with moderate hyperkalemia are relatively low (sensitivity 13.3–18.3%)7, AIDE support may be most beneficial for assisting in the diagnosis and management of moderate hyperkalemia in the clinical setting. The greater intervention effect of AIDE on ECG-moderate-hyperkalemia than on ECG-severe-hyperkalemia supports this hypothesis.

Similar to hyperkalemia, rapid correction of moderate to severe hypokalemia is advised despite the relatively lower all-cause mortality compared with hyperkalemia9. In this study, lower-than-expected AIDE intervention was achieved, and there was no significant difference in hypokalemia-related treatment between the intervention and control groups. A possible explanation for these unexpected findings involves the reduced power of hypokalemia-related ECG changes to serve as clinical cues when compared to ECG changes associated with hyperkalemia; therefore, most physicians still rely on laboratory confirmation in the clinical setting. Additionally, previous studies have reported a nonsignificant benefit of correcting hypokalemia, even to a severe extent (Lab-K+ ≤2.5 mmol/L)22. Nevertheless, hypokalemia may be more hazardous than hyperkalemia in CKD patients30. Future research on AI-ECG for hypokalemia may focus on specific populations rather than broad applications in the ED setting.

With respect to safety outcomes, overtreatment leading to the opposite form of dyskalemia may be the most undesirable side effect31. In this RCT, there was no difference between the intervention and control groups in terms of the risk of treatment-induced hypokalemia and hyperkalemia events. More importantly, treatment-induced hypokalemia in the ECG-hyperkalemia subgroup was mild, with a significantly lower incidence (only 1.3–2.5%, often developing after hemodialysis) than that reported in a previous study (6.5%)32. As mentioned above, the treatment rate of 69.1% after AIDE alerts in the ECG-hyperkalemia subgroup was significantly higher than the positive predictive rate of 43.6% in this trial. The lower overtreatment rate observed in our trial may also be attributed to the Hawthorne effect, whereby participating physicians, aware of being monitored as part of the study, provided enhanced care to their patients33. Importantly, post hoc analysis confirmed that the majority of hyperkalemia-related treatments were appropriate, and most patients who did not receive treatment within three hours eventually received hyperkalemia-related treatment. Although the AIDE alert effectively prompts timely treatment, it is still important to note that such an approach might also lead to an increase in unnecessary interventions and diagnostic cascades, as reported in other settings of substantially increased overdiagnoses34 and healthcare costs35 without clear benefit.

No significantly improved clinical outcomes were reported in any prespecified secondary analyses, which might be due to the limited number of ECG-dyskalemia subgroups. Moreover, on the basis of the wide individual variations in the management of dyskalemia36,37, the effects of AIDE intervention on clinical outcomes might be largely dependent on each physician, as reported in other RCTs using EHR-based intervention38. Given the insufficient sample size to assess the impact of AIDE on clinical outcomes in this trial, we look forward to conducting a larger transnational multicenter RCT to validate the benefit of this digital care model.

Although AIDE may yield false-positive results39, our previous and recent studies revealed that patients with pseudopositive AI-ECG dyskalemia are associated with subsequent adverse events8, as well as an increased risk of mortality40. Since pseudopositive ECG-dyskalemia is associated with a higher risk of cardiac events, it is highly likely that emergency physicians will proactively engage in cardiac risk evaluation upon receiving an AIDE alert. Although our trial was not designed primarily to evaluate these ancillary diagnostic steps, the post hoc data suggest that the alert’s reference to an elevated cardiac risk may encourage physicians to perform further cardiac examinations. This increase in cardiac evaluations aligns with the widely recognized benefit of using AI to uncover ECG findings that may have gone unnoticed by human clinical experts, such as low ejection fraction41, paroxysmal atrial fibrillation42, and the underlying cause of hypokalemia43. Whether heightened cardiac assessment after AIDE alerts reduce morbidity and mortality requires further validation.

In the ECG-hyperkalemia subgroup, our findings suggested that physician demographics might play a role in the adoption of AI-driven decision-support tools. Younger physicians and resident doctors—who are presumably more accustomed to integrating digital resources into clinical workflows—showed a slightly greater propensity to act on the AIDE alert. These results aligned with those of previous studies, indicating that early-career clinicians are more open to medical technologies44. The predominance of resident doctors among participants in this study might also be a contributing factor to the trial results. Collectively, these observations also underscore the value of future research on AI-ECG applications to focus on physician characteristics.

It is important to acknowledge the limitations of this study. First, this RCT was designed in a single-blind fashion, which might introduce bias in physicians’ behaviors and treatment decisions. However, it was impractical to blind ED physicians to AIDE alerts. Second, we assumed that all the enrolled physicians complied with the AIDE alerts and performed appropriate interventions; however, the responses provided by different physicians varied. We did not capture detailed log data to confirm that clinicians actively acknowledged the AIDE alert; therefore, while we believe that the prominent display of the alert ensured its visibility (Supplemental Note 1–Study Protocol Fig. 1), we cannot quantify the extent of its active recognition by individual clinicians. Third, although the risk of overtreatment was low in this trial, it was still unclear whether all the treatments were appropriate. Fourth, the ECG tests were triggered for clinical purposes but not for all ED patients. There may be a larger number underdiagnosed dyskalemia patients without ECG, and the effectiveness of AIDE might be highly dependent on the examination policy. Moreover, patients with overt severe dyskalemia were excluded because they received hyperkalemia-related treatment before ECG (n = 5), which could lead to a slight underestimation of the prevalence of dyskalemia. Fifth, in this RCT, we investigated the effect of AIDE for decision support among physicians, but the impact on clinical outcomes was not fully evaluated. Sixth, Lab-K+ was not available for accurate analysis in many dyskalemia patients because of the pragmatic protocol. Consequently, the positive predictive value for hyperkalemia might be underestimated. Seventh, the absence of a formal qualitative survey of physicians’ experiences with the AIDE system led to no additional insights into usability, workflow integration, and perceived clinical value. Finally, most patients in this RCT were enrolled from a single academic medical centre, while a few patients were enrolled from another community hospital. Moreover, female physicians made up a minority of the sample because of the nature of the military hospital environment. Our AIDE alert tool was developed using data collected from the same institution, making its performance closely tied to the specific demographics and data quality of the dataset, potentially limiting the generalizability of our findings to other clinical settings.

Overall, the integration of AIDE into EHR effectively alerts physicians to provide timely assessment of high-risk patients with dyskalemia, facilitating prompt treatment for hyperkalemia. However, no significant improvement in clinical outcomes was observed in this short pragmatic RCT. This trial presents a potential framework for incorporating more AI algorithms into EHRs, thus fostering increased human–machine collaboration in future clinical practice. Large-scale RCTs should be conducted to further validate the improvement of dyskalemia care, particularly in terms of clinical outcomes.

Methods

Trial registration and design

We registered this parallel, open-label pragmatic RCT with a 1:1 allocation ratio between the intervention and control groups on 11 November 2021 (ClinicalTrials.gov number NCT05118022; https://clinicaltrials.gov/study/NCT05118022) and followed the guidelines outlined in the CONSORT45 and CONSORT-AI Extension46 checklists during study reporting. The original agreement has not been changed, and the study protocol is presented in Supplemental Note 1.

Ethical statement and eligibility criteria

The institutional review board (IRB no. B202305062) of Tri-Service General Hospital, Taipei, Taiwan, conducted an ethical review for this EHR-based trial and approved the study, considering that ED physician consent, not patient consent, was needed. Eligible participants included all physicians working in the ED, including physicians providing temporary support. We explained the AIDE project on 12 November 2021 to emergency physicians authorized to develop treatment plans for patients under their care, and 70 physicians provided written informed consent. This briefing provided an overview of the operational mechanism of the AIDE alert system (including the pop-up alert), detailed the treatment strategies under consideration, highlighted its positive predictive value of ~40%, and explained the association between pseudopositive results and future cardiovascular events.

Since the research team did not have direct contact with the patients and relied on EHRs to collect patient data, the institutional review board granted a waiver of patient informed consent according to four universally recognized principles47: (1) participation poses no greater risk to the patients—according to guidelines provided by the Taiwan Food and Drug Administration (TFDA) (no. 1101603684), this condition was met as long as long as the ECGs used for AI analysis were legally obtained as part of routine clinical care; (2) investigations could not otherwise be carried out—the primary objective of this trial was to assess the impact of AI-ECG on healthcare quality, which was targeted to all ED patients instead of a specific population; (3) waiver or alteration of informed consent does not adversely affect the rights or welfare of participants—all participating physicians provided standard care to patients in accordance with existing clinical guidelines; and (4) additional pertinent information is provided to participants when appropriate—the participating physicians and patients both engaged in medical decision-making and shared all relevant information, including the results of the AI-ECG, which was recorded in the medical records. The AIDE alert system evaluated incoming ECGs using fixed, pretrained model weights; at no point during the trial were patient data used to retrain, fine-tune, or update the AI model. All the analytic scripts were run on the secure server, and no dataset left the hospital’s internal network.

This pragmatic RCT, starting on 1 January 2022, was conducted at an academic medical center and a community hospital belonging to the Tri-Service General Hospital Healthcare system. The AIDE support system was available for physicians in the intervention group. Since the COVID-19 pandemic significantly affected daily practices from 1 May 2022 to 31 August 2022, and the required sample size was not reached at that time, we temporarily withheld the AIDE support system and restarted this trial on 1 September 2022. Finally, the required number of patients was fulfilled during the monthly review on 31 October 2022, and the trial was thus terminated.

Randomization

Prior to the start of this trial, we completed the randomization process using simple sampling at random, and half of all the participants were allocated to the intervention group. Randomisation was conducted at the physician level, 35 and 35 physicians were allocated to the intervention and control groups, respectively. The hospital’s information management office engineers generated the computer-randomised allocation sequence, and enrollment, along with assignment to interventions, was automatically executed via the hospital information system, allowing physicians to immediately determine whether they would receive the AI-ECG report upon logging in. Patient data were grouped according to the physician’s assigned group. For the analysis, only patients aged ≥18 years with at least one ECG were included. Patients who received dyskalemia-related treatment prior to the ECG examination and ECGs performed during the inactive period of the AI-ECG system were excluded.

Randomization was conducted at the physician level rather than at the patient level to accurately reflect the real-world workflow in the ED. Under this design, each physician in the intervention group received an AI-ECG alert for all of their patients, while physicians in the control group did not. This approach eliminated the need for physicians to determine on a case-by-case basis whether a given patient was in the control group or if the AI data should be ignored—a task that is impractical in the fast-paced ED setting. Additionally, this design ensured that the intervention was uniformly applied within each physician’s practice, thereby isolating the impact of the AI alert on clinical decision-making regarding dyskalemia treatment.

AIDE intervention and blindness

The details of the AIDE system have been described previously8. We focused on moderate-to-severe hyperkalemia (Lab-K+ ≥6 mmol/L) and hypokalemia (Lab-K+ ≤3 mmol/L) because of the emergent nature of these conditions25. When the AIDE system detects a high risk of dyskalemia, a conspicuous pop-up alert appears in scarlet on the EHR interface of the corresponding physicians in the intervention group. The alert message was designed as follows: “An ECG from patient X indicates a high risk for hyperkalemia/hypokalemia. The estimated ECG-K+ is X. Please assess patient data to arrange suitable treatment. According to the previous study, you may also need to assess the risk of cardiac events. Please click on the following link to connect the ECG and the result of AI-ECG prediction.” The estimated ECG-K+ value allowed physicians to assess the severity of dyskalemia and to make informed treatment decisions, and physicians in the intervention group could review AI-ECG predictions both in patients with and without AIDE alerts. AIDE alerts also reported that patients with pseudopositive ECG-dyskalemia had a much higher risk of cardiac events. In the control group, physicians could not see the AIDE alert and AI-ECG reports during this trial. Severe ECG hyperkalemia (Lab-K+ ≥6.5 mmol/L) and ECG hypokalemia (Lab-K+ ≤2.5 mmol/L) were also predefined. To avoid alert fatigue, we selected cutoff points with expected positive predictive values of ≥40% on the basis of previous data8 and the consensus of the enrolled physicians. The participating physicians were aware of the cardiac risks associated with ECG-related dyskalemia.

ED clinical workflow in this trial

To illustrate the real-world workflow in our ED, we incorporated the flow diagram shown in Fig. 5. Panel (a) depicts the general ED process, beginning with arrival and triage. During triage, patients presenting with higher acuity (e.g., triage levels 1 and 2) or specific symptoms (e.g., chest pain) may undergo an immediate ECG. This is followed by physician assessment, at which point laboratory tests (such as K+ measurement) based on clinical judgment may be ordered. Finally, after reviewing the findings, the physician issues appropriate medical orders accordingly and generates an examination report as needed. Although some patients might receive an ECG later in their ED stay (for example, if initial triage and initial physician assessment did not indicate the need for ECG examination), the majority of ECGs were performed within one hour of arrival (93.6%).

Fig. 5: Flow diagram of the emergency department workflow.
Fig. 5: Flow diagram of the emergency department workflow.
Full size image

a A flow chart of the emergency department process, starting with the ED visit and triage. Depending on the triage level and symptoms, an ECG may be performed prior to physician assessment. The physician then evaluates the patient and, as appropriate, orders further tests, including potassium measurements. This may lead to additional medical orders and an examination report. b Four examples of different patient timelines for hyperkalemia demonstrate how ECG, laboratory tests, physician assessments, and AI-ECG alerts (AIDE) can occur in different scenarios. Patient 1 undergoes an ECG and laboratory test around the same time, with treatment prior to the laboratory report; Patient 2 receives an ECG and is treated on the basis of the AI-ECG alert prior to laboratory testing; Patient 3 receives an ECG and is treated on the basis of subsequent laboratory testing and reports; Patient 4 undergoes a laboratory test with hyperkalemia and is treated after the AIDE alert.

Panel (b) shows four examples of patient timelines—Patient 1, Patient 2, Patient 3, and Patient 4—which highlight how ECGs, laboratory tests, and physician assessments can occur in varying sequences within a pragmatic ED setting. Patient 1 shows a scenario where the ECG is performed early, and a lab test is ordered concurrently or shortly thereafter. Patient 2 illustrates that the physician may initiate treatment on the basis of the AI-ECG alert before obtaining the Lab-K+. Patient 3 shows treatment after Lab-confirmed K+ reports, which were more common in the control group since the AIDE alert was unavailable. Of course, physicians in the intervention group could also choose to ignore or not accept the results of AIDE to achieve a similar process. Patient 4 represents a different scenario: these individuals were deemed not to require an ECG during their initial assessments and only underwent ECG after their Lab-K+ levels were confirmed. Notably, any patient who received treatment before the ECG was excluded (five patients in total, as shown in Fig. 1). These examples underscore the inherent variability in ED workflows, where physicians balance immediate clinical action with available diagnostic information.

Baseline characteristics

We analyzed only the first ECG for each patient and defined the index time accordingly. The baseline characteristics were collected from the EHR. The central laboratory report of the Lab-K+, including the blood sampling time and report release time, was recorded. Conditions, including CKD, diabetes mellitus, hypertension, hyperlipidemia, coronary artery disease, and atrial fibrillation, were determined using corresponding International Classification of Diseases (ICD) codes. We also collected hemodialysis records (if available) to distinguish dialysis-dependent patients.

Prespecified primary, secondary, and safety endpoints

The prespecified primary endpoints for patients with ECG hyperkalemia were the timing of intervention and hyperkalemia-related treatment, including intravenous administration of calcium chloride, insulin with glucose, sodium bicarbonate, loop diuretics, nebulized or inhaled β2-agonists, oral K+ ion-exchange resin, and hemodialysis. For hypokalemia-related treatment, the timing and intravenous or oral K+ supplementation with any K+ formula (K+ chloride, gluconate, and citrate) were recorded. We considered only treatments that occurred within three hours following AIDE alerts.

The prespecified secondary endpoints were clinical outcomes, including cardiac arrest within three days, all-cause mortality within 365 days, electric shock within six hours, and CPR events within 6 hours. The prespecified safety outcomes included the side effects of dyskalemia-related treatment. The most important side effects were iatrogenic conditions, including overtreatment-induced hypokalemia (Lab-K+ ≤3.5 mmol/L) for ECG hyperkalemia and overtreatment-induced hyperkalemia (Lab-K+ ≥5.5 mmol/L) for ECG hypokalemia events within 24 hours. We excluded dyskalemia events without treatment.

Sample size

For hyperkalemia, we assumed that 40% of patients with ECG hyperkalemia (the expected positive predictive value of AIDE) should receive immediate treatment and that less than 50% of them (the sensitivity of physicians) might be identified by physicians in the absence of AIDE support7. Therefore, AIDE alerts might increase hyperkalemia-related intervention from 20% to 40%. Since hypokalemia may be less urgent than hyperkalemia, AIDE alerts might increase hypokalemia-related intervention from 10% to 20%. We performed sample size estimation using a significance level of 0.05, a statistical power of 0.80, and a sample size ratio of 1.0 for the intervention and control groups, and the minimum numbers were 91 and 219 per arm for hyperkalemia and hypokalemia, respectively. Since our preliminary study revealed that the prevalence of ECG hyperkalemia and ECG hypokalemia was 1.2% and 3.8%, respectively8, the number of samples required per arm for hyperkalemia and hypokalemia were 7583 and 5763, respectively. Therefore, the required sample size was 15,116, which was calculated a priori.

Prespecified statistical analysis plan

A two-sided significance level of p < 0.05 was set in this RCT analyzed by R version 3.4.4. The means and standard deviations or percentages were used to present the patient characteristics stratified by randomization and AIDE prediction, respectively, and the Student’s t test, analysis of variance, or the chi-square test was used accordingly. A Cox proportional hazard mixed effect model was used to compare the intervention group to the control group for the primary and secondary endpoints, with the enrolled physicians as random effects, which was conducted using the R package “coxme” version 2.2–18.1. Hazard ratios (HRs) and 95% confidence intervals (95% CIs) were used as effect indicators, and Kaplan–Meier curves were used to visualize and calculate the cumulative incidence of events. To visualize the relationship between ECG-K+ and clinical decisions, we fitted a Cox proportional hazard mixed effect model stratified by the intervention and control groups, and the “pspline” function in the “survival” package was applied to demonstrate the nonlinear relationship. Since various treatments might act concurrently as dyskalemia-related treatments (for example, insulin not only lowers potassium levels but also treats hyperglycemia), analysis of the entire sample might be confounded by treatments that were not specifically targeting dyskalemia. Therefore, the prespecified primary and secondary analyses included subgroup analyses for the ECG-hyperkalemia, ECG-hypokalemia, and ECG-normal groups.

Prespecified exploratory subgroup analyses were performed on the basis of study site, age, sex, and baseline comorbidities. Moreover, we aimed to determine the laboratory data availability simultaneously with the time when physicians received (or did not receive) the AIDE alert, particularly in the ECG-hyperkalemia subgroup. We anticipated that some patients would have a laboratory report prior to the ECG and attempted to analyze the impact of a nonhyperkalemia laboratory report on the AIDE alert intervention. The findings offered additional insight into the interplay between lab availability, AIDE alerts, and clinical decision-making in a pragmatic ED environment.

Post hoc statistical analysis

Since Lab-K+ played a key role in these treatments, we also analyzed the accuracy of AIDE using Lab-K+ collected within one hour compared with the index time before dyskalemia-related treatment. The ROC and precision–recall ROC (PRROC) curves were used to evaluate AIDE performance. We also compared the severity of ECG-related dyskalemia (ECG-K+) and Lab-K+ via the Pearson correlation coefficient (r) and mean absolute error (MAE). The Bland–Altman plot describes the mean difference and 95% interval.

Each treatment for hyperkalemia management, including intravenous administration of calcium chloride, insulin with glucose, sodium bicarbonate, and/or loop diuretics, nebulized or inhaled β2-agonists, oral K+ ion-exchange resin, and hemodialysis, was compared between the intervention and control groups using a Cox proportional hazard mixed effect model. Stratification by severity was performed for both the ECG-moderate-hyperkalemia (ECG-K+ 6.0–6.4) and ECG-severe-hyperkalemia (ECG-K+ ≥6.5) subgroups, and the same stratified analysis for the moderate (ECG-K+ 2.6–3.0) and severe hypokalemia (ECG-K+ ≤ 2.5) subgroups was also performed.

In the post hoc analysis of patients who received hyperkalemia-related treatment before the formal Lab-K+ report, comparisons between the intervention and control groups were performed using chi-square tests for categorical outcomes, such as the proportion of patients meeting treatment criteria (Lab-K + ≥ 5.5 mmol/L). For patients without hyperkalemia-related treatment within three hours, the final Lab-K+ levels were analyzed to assess the effectiveness of early treatment interventions.

In real-world ED settings, treatment decisions may be influenced by both the AI-ECG report and subsequent laboratory test results. Notably, in situations where both data sources are available, each contributes only partially to the clinical decisions. To delineate treatments administered before and after laboratory reporting using a Cox proportional hazard mixed effect model, we were able to isolate the effect of the AI-ECG on early treatment decisions.

To determine the impact of the pseudopositive AIDE score on subsequent diagnostic evaluations, we compared the frequency and timing of additional cardiac and laboratory assessments between the intervention and control groups. Owing to the limited availability of Lab-K+—particularly in the intervention group—and the even smaller number of patients with normal Lab-K+ values, further stratification to isolate pseudo-positive cases would have resulted in small sample sizes. Additionally, in the ED setting, physicians may not wait for Lab-K+ results to become available before arranging cardiac examinations. Thus, it is likely that additional cardiac tests were ordered on all patients, regardless of whether they belonged to the pseudo-positive subgroup. Consequently, analysing all patients with ECG—hyperkalemia was reasonable for this post hoc analysis. We performed further cardiac assessments, including troponin I, NT-proBNP, D-dimer, and echocardiogram, within suitable time windows. To compare the likelihood and timing of these events, we employed Cox proportional hazard mixed-effect models. This post hoc analysis was intended to assess whether the AIDE alert—particularly its message highlighting the risk of cardiac events in pseudopositive dyskalemia cases—influenced clinicians to order additional diagnostic tests.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.