Introduction

ST-elevation myocardial infarction (STEMI) represents the most serious acute coronary syndrome1,2. Rapid response and tailored treatment based on the patient’s condition are crucial for reducing mortality3,4,5. Although therapies recommended by guidelines have significantly improved patient outcomes, the absence of straightforward risk assessment tools has led to inefficient referral processes in basic-level hospitals that lack the capability for primary percutaneous coronary intervention (PCI). Consequently, the quality and efficiency of management after hospital arrival do not meet guideline-recommended strategies in China6,7.

Numerous risk assessment tools have been developed and contributed important insight into the association between clinical data and mortality8,9,10,11. Thrombolysis in Myocardial Infarction (TIMI) risk score8 and Global Registry of Acute Coronary Events (GRACE) score11 were well-known scores, whereas there was no Chinese involved in the construction, the lower awareness of diabetes and hypertension may reduce its accuracy12. The requisite of cardiac enzyme and serum creatinine which were usually unattainable at early medical contact make their application and implementation inconvenient. TIMI risk index9,13 and Canada Acute Coronary Syndrome Risk Score (C-ACS)10 use routinely available clinical variables, while their performance was inferior.

Risk assessment is an integral and ongoing process that should be conducted along with treatment to ensure optimal clinical management implementation3,4,5,14. Risk scores incorporate inspected results and echocardiographs and capable of extensive and comprehensive valuations could greatly enhance the accuracy15,16,17, however, using procedural characteristics as in previous risk models may cause a delay in risk assessment while awaiting the results and hinder early implementation. Additionally, the inefficient referral and management conditions of STEMI patients in China18,19 highlight the need for a risk score applicable across multiple clinical treatment stages. In contrast, most existing models cannot conduct rapid and comprehensive risk assessments simultaneously and fail to address the management challenges in developing countries18,19. Moreover, the widely applied existing scores were developed a long time ago8,9,11, and the therapeutic model changes in recent decades20,21,22 may have compromised their accuracy23,24. Consequently, constructing a scoring system that can identify high-risk STEMI patients and is easy to use both at early medical contact using simple and available variables and adaptable to the treatment progress using procedural characteristics is imperative, such a scoring system may help in improving clinical decision-making and streamlining the management.

Accordingly, we aimed to develop and validate a two-stage scoring system capable of rapid risk assessment at early medical contact and continuous risk classification throughout clinical treatment for 30-day mortality, consequently improving STEMI patient’s prognosis.

Methods

Study population and design

We developed and internal validated the two-stage risk assessment scheme utilizing the data from the Henan STEMI registry (NCT 02641262)25. In brief, it was a multicenter, observational study, patients with a definite diagnosis of STEMI and hospitalized within 30 days of symptom onset were consecutively enrolled from 66 hospitals (supplement Table 1) in central China, clinical information related to symptom onset, diagnosis, treatment and prognosis were prospectively collected. STEMI was defined following the Third Universal Definition of Myocardial Infarction (2012), those diagnosed as types 4a and type 5 STEMI were excluded. Patients enrolled were treated according to the guidelines and the study protocol was approved by the Ethics Committee of Henan Provincial People’s Hospital.

Table 1 Baseline demographics and clinical characteristics of STEMI patients according 30-day survival status in the derivation and internal validation dataset.
Full size table

From September 2016 to August 2018, a total of 5063 STEMI patients were enrolled, after excluding patients without survival status at 30 days (600 cases), and those with missing critical data (524 cases), a total of 3939 patients were included in this study. We randomly assigned them in a ratio of 7:3 to derivation (N = 2757) and internal validation (N = 1182) datasets using the simple random sampling method. To assess the reproducibility and generalizability of the risk scores, an independent cohort consisting of 1315 STEMI patients enrolled from 21 sites (8 tertiary hospitals and 13 secondary hospitals) (supplement Table 2) in central China between November 2019 and March 2020 was used for external validation (Supplement Fig. 1).

Table 2 Univariate and multivariable analysis for variables selected as predictors for the two-stage scoring system.
Full size table
Fig. 1
figure 1

The two-stage scoring system. SBP, systolic blood pressure. HR, heart rate. BPM, beats per minute. MI, myocardial infarction. LVEF, left ventricular ejection fraction.

Full size image

Data collection and definitions

Clinical data were prospectively collected using a web-based and password-protected data collection platform, which was capable of real-time automatic logic and range check on the completeness and validity of the data and automatically reminds investigators of 1-month, 6-month, and 1-year follow-ups. The included patients were assigned with a unique ID identified through the patients’ ID number to avoid duplicate input. The investigators of each participating center received a detailed training program on protocol, data collection, and software system before patient enrollment, and the investigator meetings were annually held to strengthen training. The study authors regularly check the consecutiveness and data quality and send queries to participating sites to review and revise, and a total of 53.84% of reported cases were audited for accuracy against medical records for onsite quality control25. Data collected included demographics, cardiovascular risk factors, medical histories, clinical characteristics at admission, reperfusion therapy, medications, laboratory examination, and echocardiography. The history of coronary heart disease was defined as having a clinical history of myocardial infarction (MI) or undergoing PCI or coronary artery bypass grafting before the current hospitalization. The anterior myocardial infarction was determined by an electrocardiogram.

Outcome assessment

The endpoint of the two-stage scoring system was all-cause death within 30 days since the first medical contact (FMC). We thoroughly reviewed hospital records for state at discharge, and all the surviving patients at discharge were followed up through telephone or clinic interviews by contacting the patients or their first-degree relatives to confirm their state at 30 days.

Statistical analysis

The patient’s characteristics, treatment therapy, inspected results, and electrocardiographic were compared between survivors and non-survivors in derivation and validation datasets. Categorical variables were presented as numbers and percentages, Chi-square or Fisher exact tests were used for comparisons as appropriate, whereas continuous variables were reported as means and standard deviation (SD) or median and interquartile range (IQR), t-test or Mann-Whitney U test was used as appropriate.

The two-stage scoring system was constructed by fitting demographic and clinical characteristics, inspected results, and electrocardiographic variables, which were selected based on the findings of previous studies and their clinical significance. In consideration of the variable availability on early medical contact, inspected results and electrocardiographic variables were not considered for entry into the FMC stage risk score. Univariate Cox proportional-hazards regression was used for potential risk factors screening, and multivariate Cox proportional-hazards regression was used for independent predictors identifying, the variables with a P < 0.05 in the multivariate analysis were entered into the final model.

To simplify the use of the two-stage scoring system in clinical practice, continuous variables were converted into categorical variables, and we developed the two-stage scoring system based on the hazard ratio (HR) estimated from the final multivariate model, and the integer score was generated by rounded the HR of each variable up to the nearest integer. The risk score was calculated by adding together all the points corresponding to variables and stratified patients into three classifications based on the summed score and the average predicted 30-day mortality: low (the predicted mortality rate was less than 5%), intermediate (the predicted mortality rate was 5–15%) and high (the predicted mortality rate was greater 15%).

The area under the curve (AUC) was calculated and compared with the TIMI risk score, GRACE score, TIMI risk index, and C-ACS using a nonparametric test developed by DeLong. The Hosmer-Lemeshow (HL) test and the Nagelkerke-R2 from the regression modeling were used as indicators of goodness-of-fit and calibration ability, and the Brier scores were also calculated. The absolute integrated discrimination improvement (IDI) and category-free net reclassification improvement (NRI) were used to evaluate improvements in risk predictions quantified compared with the classic risk score. Decision curve analysis (DCA) was conducted to determine the clinical usefulness by quantifying the net benefits at different threshold probabilities. Finally, we applied the two-stage risk assessment scheme to validate datasets to further evaluate their stability.

Statistical analyses were performed with SAS 9.4 (SAS Institute Inc., Cary, NC) and R package (Version 4.2.1, R Foundation for Statistical Computing, Vienna, Austria). Two-sided P values < 0.05 were considered statistically significant.

Results

Study patient’s characteristics

A total of 3939 patients were included, 352 (8.9%) deaths occurred within 30 days after FMC, including 250(9.1%) in the derivation datasets and 102(8.6%) in the internal validation datasets (supplement Fig. 1). As shown in Table 1, except for peripheral arterial disease and reperfusion strategy, there were no significant differences between derivation and internal validation datasets.

Both in the derivation and internal validation dataset, the non-survivor had high proportions of older age ( > = 75), female, a history of dyslipidemia, diabetes, current smoker, and heart failure. Furthermore, patients who died within 30 days were more likely to present with anterior MI and Killip class II and above, and these patients had lower systolic blood pressure (SBP), higher heart rate (HR) and were less likely to receive reperfusion strategy, the Onset-to-FMC time was significantly prolonged, what’s more, these patients had lower proportions of LVEF ≥ 50% and serum creatinine < 100 umol/L (Table 1).

Development of the two-stage scoring system

Eight significant factors including age, gender, SBP, HR, Killip class, anterior MI, LVEF, and serum creatinine emerged as predictors of 30-day mortality through univariate and multivariate analysis in derivation datasets (Table 2). In consideration of their availability on FMC, we took six predictors including age, gender, SBP, HR, Killip class, and anterior MI into the FMC stage prediction model, LVEF and serum creatinine were added to the in-hospital stage prediction model.

According to the HR and corresponding 95% confidence interval of the categorized predictor in the derivation datasets (Table 2), scores were assigned to each predictor as shown in Fig. 1. Considering the convenience of constant evaluation, the scores for “Age > = 75” and “SBP < 90 mmHg” in the FMC stage prediction model were modified to align with the in-hospital stage prediction model (supplement Table 3), and the modification of FMC stage risk score did not affect the discrimination and calibration ability (supplement Table 4), and the prognostic performance was equivalent after modification (supplement Table 5).

Table 3 Prognosis performance of the two-stage scoring system for the prediction of 30-day mortality.
Full size table
Table 4 Comparisons of the prognostic performance of the two-stage scoring system with GRACE, TIMI risk score, TIMI risk index and C-ACS for 30-day mortality.
Full size table

The sum of the score, which could theoretically range from 0 to 19 for the FMC stage risk score and range from 0 to 28 for the in-hospital stage risk score, could be used to estimate the risk of 30-day mortality. The obtained scores ranged from 0 to 17 for the FMC stage and ranged from 0 to 23 for the in-hospital stage (Fig. 1). The two-stage scoring system displayed good discrimination (supplement Fig. 2a) and calibration ability (Table 3, supplement Fig. 2b, supplement Fig. 2c).

Fig. 2
figure 2

ROC comparison of the two-stage scoring system with TIMI risk score, GRACE risk score, TIMI risk index, and C-ACS. (A) ROC comparison in derivation datasets. (B) ROC comparison in internal validation datasets.

Full size image

Based on relationship between scored points and the probability of 30-day mortality (supplement Fig. 3), the STEMI patients were further categorized into the following 3 classifications in the FMC stage: low risk (score < = 5), intermediate risk (score 6–8) and high risk (score > = 9), and the observed incidence was 2.3%, 7.5%, and 25.7%, in the in-hospital stage: low risk (score < = 8), intermediate risk (score 9–11) and high risk (score > = 12), and the observed incidence was 1.9%, 11.0%, and 32.4% respectively (Fig. 1, supplement Fig. 4a). The intermediate and high-risk patients in the two-stage scoring system experienced higher incidences of 30-day mortality than their counterparts (supplement Fig. 5a, supplement Fig. 5b).

Fig. 3
figure 3

Decision curve analysis(DCA) of the two-stage scoring system, TIMI risk score, TIMI risk index, GRACE risk score, and C-ACS for 30-day mortality. (A) DCA in derivation datasets, (B) DCA in internal validation datasets. The x-axis indicates the threshold probability, the y-axis indicates the net benefit, the yellow line displays the net benefit of the strategy of treating all patients; the pink line illustrates the net benefit of the strategy of treating no patients.

Full size image

Internal validation of the two-stage scoring system

The two-stage scoring system displayed good discrimination and calibration ability in the internal validation dataset (Table 3; Fig. 2b). According to the sum of scores, the STEMI patients in the internal validation datasets were also categorized into 3 classifications, the event rate was 1.3%, 7.2%, and 26.2% in the FMC stage, and 1.5%, 8.9%, and 35.8% in the in-hospital stage (supplement Fig. 4b) for low-risk, intermediate-risk and high-risk patients respectively, the intermediate and high-risk patients experienced significant higher incidences of 30-day mortality (supplement Fig. 5c, supplement Fig. 5d).

Comparison with the classic risk scores

In the derivation datasets, the AUC of the FMC stage risk score was superior and observed an improved IDI compared with C-ACS, and no significant AUC differences and significant IDI were observed compared with the other scores (all P > 0.05). What’s more, the NRI of the FMC stage risk score was significantly enhanced compared with the TIMI risk index and C-ACS (all P < 0.001). As for the in-hospital stage risk score, an improved AUC, NRI, and IDI were observed compared with the classic risk scores (all P < 0.001) (Table 4; Fig. 2). The DCA showed that the in-hospital stage risk score added more benefit than the classic risk scores in the threshold probability interval of 0 to 0.50, and the FMC stage risk score added comparable net benefit compared with the TIMI risk score and GRACE score, and showed more benefit than TIMI risk index and C-ACS (Fig. 3a).

In the internal validation datasets, the AUC of the FMC stage risk score was better than the TIMI risk index and observed an improved NRI compared with the TIMI risk index and C-ACS. No significant AUC differences, significant NRI, and significant IDI were observed compared with the TIMI risk score and GRACE score (all P > 0.05). As for the in-hospital stage risk score, an improved AUC, NRI, and IDI were observed compared with the classic risk scores (all P < 0.001) (Table 4; Fig. 2). The DCA was presented in Fig. 3b, the in-hospital stage risk score showed more benefit than the classic risk scores in all the threshold probability intervals, and the FMC stage risk score earned comparable net benefit compared with the TIMI risk score and GRACE score and shown more benefit than TIMI risk index and C-ACS.

Performance in subgroups

The discrimination ability of the two-stage scoring system was exhibited after the STEMI patients were divided into subgroups according to cardiac arrest on admission, reperfusion therapy, medical history, onset-to-FMC time, hospital level, and admission and discharge in 24 h (supplement Table 6). The event rate ranged from 4.1 to 47.8% in the subgroups, and the AUCs ranged from 0.775 to 0.881 for the FMC stage risk score and ranged from 0.806 to 0.918 for the in-hospital stage risk score. Although the event rate showed differences between the subgroups, both the FMC stage and the in-hospital stage risk scores showed comparable discrimination ability and balanced sensitivity and specificity. Respectively, the two-stage risk assessment scheme had a good and stable prediction ability for 30-day mortality in various clinical characteristics and treatment settings.

External validation of the two-stage scoring system

In the external validation dataset, the non-survivor presented a higher proportion of risk factors which were included in the scoring system (supplement Table 7), and the two-stage scoring system displayed good discrimination and calibration ability (supplement Table 8). The superior AUC and improved IDI of the FMC stage risk score were observed in comparison with the C-ACS and TIMI risk index. What’s more, an improved AUC, NRI, and IDI of the in-hospital stage risk score were observed compared with the classic risk scores (all P < 0.001) (supplement Fig. 6, supplement Table 9). The in-hospital stage risk score performed more benefit than the classic risk scores in DCA analysis, and the FMC stage risk score added comparable net benefit compared with the TIMI risk score and showed more benefit than the TIMI risk index, GRACE score and C-ACS (supplement Fig. 7).

Discussion

The two-stage scoring system constructed in this study had good discrimination and calibration ability and was capable of rapid and in-depth assessment of 30-day mortality among STEMI patients. The FMC stage risk score was formed by six predictors, which were readily available at early medical contact and showed equivalent prognosis performance compared with the classic risk scores, which allowed physicians to make treatment decisions according to the rapid risk assessment at admission. The in-hospital stage risk score added serum creatinine and LVEF and obtained promotion in AUC, IDI, and NRI compared with the classic risk scores, which allows clinicians to continuously identify high-risk patients according to treatment progress and provide more targeted treatments to improve prognosis in clinical practices.

Identifying high-risk STEMI patients is the best way to prevent complications2. Various risk assessment systems such as TIMI risk score8, GRACE score11, Zwolle score26, ACEF score16, NPAC score27, SPUM-ACS Score15, and triglyceride-glucose index28 have been developed and recommended by clinical guidelines for the stratification of STEMI patients3,4,5,14, and provide useful information in determining treatment strategy. However, the TIMI risk score8 and TIMI risk index9 were derived from clinical trials and had inherent bias due to excluding high-risk patients. Although the GRACE score11 and Zwolle score26 were derived from registry study and demonstrated excellent performance, nevertheless, the scoring system was developed in an era in which PCI was not widely used20. They may not be as accurate for STEMI patients undergoing PCI23,29, and their practical application is somewhat impaired by the requisite post-procedural TIMI flow and the complex formula that requires computer-assisted calculation. Meanwhile, Chinese patients and patients in developed countries differ in terms of the therapeutic scheme, clinical features, physique, and awareness of cardiovascular risk factors, for instance, the proportion and efficiency of reperfusion therapy were still low among Chinese STEMI patients19,21,22, and almost two-fifths of STEMI patients initially arrive at hospitals with limited ability and need transfer to primary percutaneous coronary intervention (PCI)-capable hospitals, yet the transfer was inefficient and the transferred patients experienced prolonged time delay18, which detrimentally affecting prognosis. Rapid risk stratification after first medical contact might help in decision-making and improving transfer efficiency. However, to enhance accuracy, the existing scores incorporating laboratory results such as NT-proBNP15, serum creatinine11,16,27, neutrophil-to-lymphocyte ratio27, triglyceride and glucose28, as well as cardiac ultrasonography parameters such as LVEF15,16,30 into the model, and were incapable of stratifying risk of death at early medical contact. What’s more, neither Chinese nor other Asians took part in the construction of the classic scoring systems8,9,10,11,15,16,26, and whether the risk scoring systems devised in the developed countries were applicable to Chinese STEMI patients was doubtful31. Consequently, a risk assessment scheme that complements the existing classic scores and is aligned with contemporary clinical practice in China is needed. The two-stage scoring system in this study was derived from a representative registry of Chinese STEMI patients with standardized data collection, and the abstracted information was rigorously audited and was likely to be suitable for representing current clinical practice in China25.

The prognosis of STEMI patients was affected by many variables, there was no single predictor can predict mortality, and the more variables in the scoring system identified the better the predicting effect17,32,33, meanwhile, incorporating more variables not only prolongs the evaluation progress but also potentially limits its practicality, particularly early after hospital admission. Risk assessment schemes at the initial point of medical contact should not aim to capture all variables influencing prognosis but should provide an accurate preliminary risk estimate. Although there have been risk scores capable of predicting prognosis at early medical contact, whereas, the score was not specific to STEMI patients10,34, and it takes in-hospital death as endpoints, which ignores the actual situation that most Chinese patients are reluctant to die in the hospital6, and the 30-day mortality can better exhibit the risk of death during acute phase. The FMC stage risk score in this study shares 4 independent predictors (age, SBP, HR, Killip class) with previous risk scores8,10,11, gender, and the location of MI were newly introduced. Female STEMI patients experience a higher risk of short-term mortality than their counterparts, and gender was shown to be an independent predictor of mortality even after adjustment for age and comorbidities35,36. Previous studies indicated that the location of MI was also an important and independent predictor37, the anterior MI increases the risk of death and post-infarction heart failure38. Although some studies have demonstrated that the standard modifiable cardiovascular risk factors (SMuRFs, hypertension, diabetes, hypercholesterolemia, and smoking) were associated with adverse outcomes39, due to the low awareness and availability at emergencies12, SMuRFs were not included in FMC stage risk score. The six variables included were readily available, and the FMC stage risk score displayed equivalent predictive ability compared with the classic scores. In addition, the FMC stage risk score exhibited stable discrimination ability for different subgroups and was a useful practical tool, especially for primary medical facilities with limited medical resources.

The association between renal function and echocardiography parameters with adverse clinical outcomes has been demonstrated in previous studies16,27,30,40, adding them to the prediction model could improve the predictive capacity in STEMI patients15,27,41. However, the blood test and echocardiography results demanded extra time and effort. Furthermore, despite the guidelines recommending echocardiography at early medical contact and upon discharge3,4,5, serum creatinine and echocardiography were lacking for some patients in clinical practice6,42. Consequently, these parameters were not included in the FMC stage risk score. With the progress of the clinical treatment, these independent predictors were available and made progressive evaluation possible, the clinician can provide preferable attention and evidence-based longitudinal care for STEMI patients. Although the creatinine clearance rate and estimated glomerular filtration rate were better renal function indicators than serum creatinine40, to simplify the calculations, serum creatinine as a categorical variable was included in the in-hospital stage risk score. LVEF was unavailable for 35.1% of STEMI patients, primarily due to their relatively adverse health status, which resulted in the reluctance for further tests. The absence of LVEF measurements was an important predictor30 and was included for further evaluation. Our results indicated that the in-hospital stage risk score performed well on discrimination and calibration capability. The significantly improved AUC, NRI, and IDI were observed compared with the TIMI risk score, GRACE score, TIMI risk index, and C-ACS. The in-hospital stage risk score was more effective in predicting 30-day mortality than these four classic scales. Additionally, in the present study, we applied the two-stage risk assessment scheme in different subgroups and showed high consistency.

The two-stage risk assessment scheme constructed in this study is straightforward to calculate and performs superior evaluation capabilities compared to the classic scores, different from the scoring system implemented so far, the scoring system in this study is capable of identifying high-risk patients with 30-day mortality at early medical contact and with the progress of the clinical treatment, which is consistent with the clinical practice in China and aims at the demand of current management difficulties18,19,21,22, and can be used to remind the physician that the patients were at high risk of 30-day death that threaded throughout the clinical treatment and may play an important role in promoting more attention management regarding therapeutic strategies and medical resources allocation. The FMC stage risk score was capable of rapid evaluation and is likely to be a suitable candidate tool for shortening the time of shared decision-making and improving the efficiency of inner-hospital transfer and the proportion of reperfusion. Furthermore, the inclusion of LVEF in the two-stage scoring system greatly enhances the accuracy, can help physicians make more extensive and comprehensive valuations, and may promote clinicians adhering to the guidelines to reinforce the application of echocardiography and the implementation of guideline-recommended therapies, which detrimentally improving prognosis.

Limitations

Our analysis had several limitations. Firstly, we only assessed the risks of all-cause 30-day mortality, we did not distinguish the specific cause of death, such as cardiac death and electrical death independent of ventricular function, and the scoring system cannot be used to predict risk of specific causes of death. Secondly, our data are observational, the information mainly comes from medical records and we only accounted for traditional clinical factors in our analysis, residual confounding factors that may have influenced the 30-day mortality failed to be adjusted and included in this scoring system. Thirdly, the data were obtained from Chinese STEMI patients, and the scoring system was mainly developed to tackle the management difficulties and improve the management situation of Chinese STMEI patients, whether the two-stage scoring system performs as well in other populations needs further investigation.

Conclusion

In summary, we developed and validated an effective and efficient two-stage scoring system to continuously quantify the risk of 30-day mortality throughout the treatment process among STEMI patients. The easily used FMC-stage risk score and the comprehensively in-hospital risk score had satisfactory discrimination and calibration ability. This scoring system can be implemented in the electronic health record or in a dashboard in patient medical records to aid in referral and management decision-making in underdeveloped counties. Further implementation of machine learning algorithms or artificial intelligence that runs dynamic risk assessment automatically on patients’ data from electronic health records is warranted to improve patient prognosis.