Introduction

Despite an overall decrease in mortality in patients discharged from intensive care units (ICU), rates of readmission in survivors have increased over the past decade, including pediatric intensive care unit (PICU) survivors1,2. Limited literature has currently identified that PICU readmissions are associated with higher mortality, a longer median length of stay (LOS), healthcare costs, and resources3,4. The mortality rate among children readmitted to the PICU ranges from 3 to 22% 2,4,5,6,7,8,9,10. Specifically, the mortality rate for readmissions within 48 h after discharge is approximately 6.6–21%2,9, and for readmissions within 7 days after discharge, it is around 6–14%6,10. These rates are higher than the overall PICU mortality range of 1.4–12%2,4,5,6,7,8,9,10. The Pediatric Data Quality System Collaborative Measure Workgroup has proposed unplanned readmission as one of the indicators to assess the quality of PICU care in 200711. Predictive modeling is an effective approach to improving the allocation of valuable medical resources by stratifying patients’ risk of readmission and providing preventive interventions for high-risk patients. There are only two published risk models for readmission to the PICU within 48 h of being transferred from the PICU to the ward or floor12,13. There is no risk prediction model available for children discharged home and readmitted directly to the PICU within 7 days. We chose to focus on 7-day readmissions because they may be a more reflective measure of healthcare quality and are potentially more preventable than the 30-day readmission rate10,14. Previous studies on PICU readmissions have focused solely on the initial 48 h after discharge2,3,12,13,15,16,17. However, readmissions within 48 h account for only 38–47% of all PICU readmissions2,15, while approximately 60% of readmissions occur more than 48 h after discharge2, with the median time to readmission being 7 days15. For these reasons, we developed and validated a clinical risk model to predict children discharged home and readmitted directly to the PICU within 7 days. Moreover, the development of a nomogram made the model clinically available to help clinicians identify and manage patients at high risk of early readmission.

Methods

Ethical statements

This study adhered to the principles of the 1964 Declaration of Helsinki. The protocol, “Development and Validation of a Risk Prediction Model for Unplanned Readmission to PICU,” was reviewed and approved by the institutional review board of the First Hospital of Jilin University (No. 2023-232; March 1, 2023). The requirement for informed consent was waived by the institutional review board of the First Hospital of Jilin University due to the retrospective nature of the study.

Patients and study design

This was a case-control study designed to develop and validate a risk prediction model. The goal of this model was to assess the possibility of patients being discharged home and readmitted directly to the PICU within 7 days between November 2011 and September 2022. The PICU at a tertiary teaching hospital in China is a 24-bed unit that admits mixed medical/cardiac/surgical patients. It is the largest one in our region, providing tertiary intensive care services for critically ill children. Our PICU has both high-dependency units and step-down units. The decision to admit or discharge patients from high-dependency units and step-down units is based on both the PELOD-2 score and the attending physician’s clinical assessment. Transfer to step-down units is considered only when the PELOD-2 score is less than 5, accompanied by the attending physician’s thorough evaluation of the child’s overall clinical condition in our center. The case group included all children aged 1 month to 18 years who required unplanned 7-day readmission to the PICU after they were discharged home from step-down unit of our PICU. The unplanned readmission was defined as children discharged home who required emergency or non-elective readmission to the PICU. For the control group, patients discharged on a day similar to the case group discharge date (no more than 3 days) were selected as controls. We identified all other patients who were discharged on that day and not readmitted. We employed individual matching to pair each case with a control based on age and sex. This brings some matches to seasonal differences in unit bed occupancy and disease categories, as well as changes in medical and care experiences. All patients included in this study had never been admitted to a PICU before. Patients who died at the time of their initial admission and lacked sufficient clinical information were excluded. Patients readmitted to the PICU for scheduled reasons, such as routine care, readmissions that happened more than 7 days ago, and patients transferred to other PICUs for further treatment, were also excluded. To mitigate the potential impact of confounding factors associated with in-hospital treatment processes, we excluded cases of PICU readmission that occurred within the same hospitalization episode.

This study followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) Guidelines18 to report the risk prediction model and assess its performance.

Data collection

For each eligible patient in the study, we collected characteristics using electronic medical records data during their first stay in the PICU. The characteristics include age, sex, home address, complex chronic conditions (yes/no), the severity of illness, the primary diagnostic category, source of admission (emergency or non-emergency), date of discharge, any surgical procedure, continuous renal replacement therapy (CRRT) requirement (yes/no), invasive mechanical ventilation (IMV) requirement (yes/no), use of sedative medications (yes/no), use of vasopressors (yes/no), the Functional Status Scale (FSS) score at discharge, hospital and PICU LOS was extracted. The time to withdraw from sedative medications, vasopressors, CRRT, and IMV was also documented.

The severity of illness was calculated using the Pediatric Logistic Organ Dysfunction 2 (PELOD2) score, which was assessed on PICU admission and within 2 days before discharge. Choosing the median of the PELOD2 score as cut-off values appeared to best divide the groups19,20,21,22. Complex chronic conditions were identified and confirmed using the definition, which was described by Feudtner and Edwards23,24. The FSS score was a kind of age-appropriate measure to evaluate the adaptive functional status of patients in the PICU25,26,27. The total FSS score classified overall function status as normal function (FSS 6–7), mild dysfunction (FSS 8–9), moderate dysfunction (FSS 10–15), severe dysfunction (FSS 16–21), and very severe dysfunction (FSS > 21)26. The validation of risk factors for readmissions within 1 year among patients has used 2 weeks of PICU LOS as a cut-off8. Sedative medications and vasopressors were administered intravenously for more than 24 h and were recorded. 48 h of the time of IMV, sedative medications, vasopressors, and CRRT withdrawal before discharge have been used in the validation of risk factors for PICU readmissions1. Two experienced investigators collected data, and all data collection was checked and overseen by the primary investigator.

Sample size

In the medical literature, the minimal sample size of the case group required is 5–10 times the maximum number of candidate predictors28,29. In our study, we planned to include 20 candidate variables, so the minimal sample size of the case group was 100–200.

Statistical analysis

Continuous data are expressed using the mean and standard deviation (SD) if normally distributed, and continuous variables are expressed as the median and interquartile range (IQR) if skewed. Categorical or grade data are expressed as frequencies and percentages. Qualitative variables were compared by the χ2 test, Kruskal–Wallis test, and Fisher’s exact test. Quantitative data of skewed distribution were compared by the Mann–Whitney U test. To screen the potential risk variables of 7-day readmission to the PICU in the developing set, we first used univariable logistic regression analysis. Risk factors that were statistically significant in the univariate analysis were included in the backward stepwise multivariable logistic regression analysis. The multivariable logistic regression analysis was performed to identify significant risk factors for 7-day readmission to the PICU and establish a clinical prediction model. The clinical prediction model for 7-day readmission risk was exhibited by a nomogram. The model was validated and demonstrated from three perspectives: discrimination, calibration, and clinical usefulness. The model’s discrimination capability was validated using the receiver operating characteristic curve (ROC) and area under the curve (AUC). The Hosmer–Lemeshow test was used for the goodness of fit test, and the calibration curves were plotted to show the model’s calibration. In the developing set, the clinical usefulness of the model was demonstrated by the decision curve analysis (DCA). Finally, the developing set was internally validated using the 10-fold crossover validation method. P-values less than 0.05 were considered statistically significant, and all statistical tests were two-tailed. The data were analyzed using Stata (V.15.0) and SPSS (V.26.0).

Results

Baseline demographic and clinical characteristics

There were 5266 children discharged home directly from our department. Of these patients, 417 (7.92%) required readmissions. 173 (3.3%) patients were included in the case group who required unplanned 7-day readmission in the final analysis. And patients in the case group were compared to 184 eligible controls. The developing set and validation set were constructed using SPSS software. The total number of patients in the case group and the control group was randomly divided into a developing set and a validation set in a widely accepted and commonly utilized ratio of 7:330,31. The developing set consisted of 256 patients, while the validation set consisted of 101 patients. And the sample size of the case group in the developing set was 128 in our study. This distribution of the overall sample provided enough power to develop and validate our risk prediction model (Fig. 1).

Fig. 1
figure 1

Study flow chart of participants.

Full size image

The baseline characteristics of the developing set and the validation set are listed in Supplemental Table 1. Characteristics had no difference among the two datasets (p > 0.05, respectively).

Risk variable selection and model establishment

Characteristics of readmitted patients and the control group in the developing set were summarized in Supplemental Table 2. In the developing set, there were no significant differences between the case group and the control group concerning age or gender (p = 0.581, p = 0.707, respectively). There was a higher proportion of patients with PICU LOS past 2 weeks in the readmitted group ([unadjusted odds ratio, OR] [95% confidence interval, CI] 1.711 [1.026–2.854]; p = 0.04). Patients who had 7-day readmission had more complex chronic conditions (unadjusted OR [95% CI] 5.129 [2.885–9.118]; p < 0.001). There was some evidence that the primary diagnostic category was also associated with readmission (p = 0.03). The requirement of vasopressors, sedation medications, and IMV during their first stay in the PICU was strongly associated with readmission (unadjusted OR [95% CI] 2.034 [1.198–3.455], p = 0.009; 3.884 [2.298–6.564], p < 0.001; and 3.537 [2.112–5.923], p < 0.001, respectively). IMV withdrawal less than 48 h before discharge was associated with readmission compared to the controls (unadjusted OR [95% CI] 2.014 [1.014–3.998], p = 0.045). The severity of illness, as assessed by the PELOD2 score on admission and discharge, was strongly significant between the two groups (p < 0.001). There was very strong evidence in the FSS categories between the two groups (p < 0.001). 7% of the patients in the case group were categorized as having severe and very severe dysfunction compared to only 2% of the controls. Patients with mild dysfunction (unadjusted OR [95% CI] 2.245 [1.208–4.172]) and moderate dysfunction (unadjusted OR [95% CI] 16.64 [2.118–130.751]) were at increased risk of readmission.

The above 10 characteristics, which were statistically significant in the univariate analysis, were analyzed by backward stepwise multivariable logistic regression analysis, and 5 risk factors with p < 0.05 were found (Table 1). The five risk factors were the presence of chronic conditions, a PELOD2 score higher than 4 on admission, a PELOD2 score higher than 0 at discharge, the use of sedation medications, and the FSS score at discharge. Based on the above five risk factors, the clinical predictive model named the “7-day Readmission Risk Assessment (7DRRA)” model was established, and the nomogram was plotted (Fig. 2).

Table 1 The multivariate analysis of unplanned 7-day readmission to the PICU in the developing set.
Full size table
Fig. 2
figure 2

Nomogram of the 7DRRA model. The scale of the line segment corresponding to each risk variable in the prediction model indicates the possible value range of the risk variable, and the length of the line segment indicates the influence of the risk variable on 7-day readmission. The point represents the individual score corresponding to each risk variable under different values, and the total score is obtained by adding the individual scores of all risk variables. The 7-day readmission prob represents the risk of early readmission in individual PICU discharged patients.

Full size image

The resulting formula of the final multivariate analysis of the 7DRRA model was Ln [p/(1−p)] = − 3.077 + 1.784 complex chronic conditions + 1.176 Use of sedation medications + 1.675 PELOD2 score higher than 4 on admission + 1.035 PELOD2 score higher than 0 at discharge + 0.53 Mild dysfunction + 2.187 Moderate dysfunction + 0.373 Severe and very severe dysfunction.

This model has a range of 0 to 35 when the five predictive factors are included. Patients with scores of 14 or higher had a greater than 50% chance of requiring early readmission, so it would be reasonable to suggest that children with scores of 14 or higher may require special attention.

Model performance

The ROC curves of the two datasets showed good sensitivity and specificity to the model (Supplemental Fig. 1). The AUCs of the developing, validation sets were 0.851 (95% CI 0.805–0.898) and 0.811 (95% CI 0.727–0.895), respectively (Supplemental Table 3). The AUCs of the developing set and the validation set both differed statistically (p < 0.001, respectively).

The Hosmer–Lemeshow goodness-of-fit test showed that the model had acceptable calibration capability (the developing set chi2 = 8.5, p = 0.580; the validation set chi2 = 14.05, p = 0.171). The calibration curve also illustrated good consistency of the model between the predicted and actual values (Fig. 3A, B).

Fig. 3
figure 3

Calibration curves of the 7DRRA model. (A) Calibration curve of the developing set. (B) Calibration curve of the validation set. The Y-axis of the calibration plot represents the actual probability of 7-day readmission to the PICU. The X-axis represents the predicted risk of 7-day readmission in the current set. The “Apparent” curve means the uncalibrated predicting curve of the 7-day readmission probability in the current set, the “Bias-corrected” curve means the calibrated curve, and the “Ideal” straight line means the perfect prediction between predicted probability and actual probability of early readmission.

Full size image

DCA showed a net benefit of applying the model to predict early readmission in patients discharged from PICU (Fig. 4). The implementation of the 7DRRA model proved helpful when PICU patients were discharged home with a risk probability of readmission within 7 days of discharge of greater than 20%.

Fig. 4
figure 4

DCA of the 7DRRA model. The Y­axis of the DCA represents the net benefit, and the X­axis represents the diagnostic threshold of the 7DRRA model. The blue line represents the assumption that all discharged children had early readmission to the PICU and would be taken for further treatment, and the red line represents the assumption that no patients had early readmission to the PICU. The green curve represents the 7DRRA model.

Full size image

Internal validation of the model

The developing set’s and internal cross-validation’s ROCs both showed high sensitivity and specificity, with AUC1 and AUC2 of 0.8514 and 0.8305, respectively, and the differences were statistically significant (p < 0.001). As a result, the risk prediction model was repeatable (Supplemental Fig. 2).

Discussion

Readmission is strongly associated with the poor prognosis of patients3,15. To support the reduction of readmissions, it would be beneficial if those at high risk could be predicted with greater accuracy. The goal of this study was to identify factors that help find those patients at greater risk of early readmission to the PICU. To the best of our knowledge, there has been no study on the assessment and identification of readmission within 7 days for PICU-discharged children. In our study, according to the clinically accessible characteristics of PICU discharged patients during their first PICU stays, we established a 7DRRA model and visualized a nomogram to make it available for each potential patient for assessment of early readmission risk, which may be a novel approach for the identification of early readmission, especially for young physicians with insufficient clinical experience.

A multicenter retrospective cohort study conducted by Jeffrey et al.16 found that critically ill children with complex chronic conditions are more likely to be readmitted, even after adjusting for other factors that can influence readmission, such as age and severity of illness. The study also indicated that the risk of early readmission for patients with complex chronic conditions was several times higher than for patients without complex chronic conditions (comparing no complex chronic conditions, adjusted hazard ratio, 1.2–1.72 for one or more than two complex chronic conditions; p < 0.001). Our results corroborate the findings from the previous studies that patients with complex chronic conditions were more likely to require early readmission.

Using the logistic regression model, we identified the use of sedatives as a risk factor for the 7-day readmission. Czaja et al.15 showed that the use of sedatives during their first PICU stays was associated with early readmission. In 2020, a retrospective observational study conducted in Japan had a similar finding10. They found that withdrawal syndrome could independently predict readmission to the PICU by multivariate regression analysis (OR 3.95; 95% CI 1.53–10.2). Because withdrawal syndrome is an adverse effect of analgesic and sedative treatment in the ICU32, its effect on PICU survivors’ readmissions can laterally confirm that sedative medicines use can increase the risk of readmission. To reduce the incidence of withdrawal syndromes and thus readmission rates, multidisciplinary involvement, and reasonable use and management of sedative medication are required.

Published papers have mentioned the relationship between the severity of illness and readmission to the PICU. Retrospective cross-sectional research reported that a Pediatric Index Mortality 2 score of 15 or higher was an independent risk factor for early PICU readmission33. Kotsakis et al.1 also found that PELOD scores at discharge were higher in readmitted patients. Our study assessed the severity of illness using the PELOD2 score, and the results demonstrated that the severity of illness was associated with early readmission both on admission and at discharge. As a component of the 7DRRA model, more weightage was given to the assessment of the PELOD2 score on admission and at discharge.

The finding that higher PELOD-2 scores at discharge were associated with increased risk of early readmission supports its validity as a predictive factor. However, this observation also raises important considerations regarding the adequacy of clinical stability at discharge. While all patients met standard hospital discharge criteria, variations in individual recovery trajectories or clinician judgment may have influenced outcomes. The higher PELOD-2 scores at discharge might reflect residual morbidity, potentially contributing to readmission risk. Future studies should explore whether optimizing discharge decisions through integrated assessments of clinical stability, social support, and post-discharge care could mitigate early readmission rates.

Critically ill patients often develop muscle weakness, physical impairments, and deterioration in functional status in the ICU, which can persist until the time of discharge or even for years after discharge34,35,36. There are limited studies analyzing the functional status at discharge associated with PICU readmission. Previous studies often used the Pediatric Overall Performance Category (POPC) score to measure the functional status of children37. A retrospective multicenter cohort analysis from the United States showed that, except for coma/vegetative states, progressively worse overall disability was associated with an increase in the odds of PICU readmission16. Kaur et al.12 also found that a higher proportion of readmitted patients had moderate to severe disabilities evaluated by the POPC score. The FSS score is more suitable for assessing the functional status in the PICU27. Our findings were similar to the above studies that children with mild and moderate dysfunction are more likely to have early readmission to the PICU than children with normal functional status at discharge (OR [95% CI] 2.245 [1.208–4.172]; OR [95% CI] 16.64 [2.118–130.751], respectively). However, children with severe and very severe dysfunction had a decreased risk of readmission to the PICU in our study. It’s probable that the number of children with severe and very severe dysfunction was limited, and that a small minority of them were discharged from the PICU and referred to other hospitals for further treatment. These factors may partly explain the lack of a significant association between severe dysfunction and readmission observed in our analysis.

There were several limitations in our study. Firstly, the study was a retrospective observational study. Although the retrospective design was likely to be exposed to recall and selection bias, the implementation process of case identification, collection, and management would reduce these flaws to the degree that is practicable in the context of epidemiological studies. Secondly, although our model was developed and internally validated, the small sample size increased the possibility of type II errors, which could decrease the model’s reliability. Thirdly, our model had no external validation, and our study used a matched case-control approach. Because each institution is different, our findings may not be suitable for another setting. Furthermore, this study is a retrospective study with a limited sample size, which to some extent restricted the matching of cases and controls. Despite our efforts to balance key variables between cases and controls, potential biases may still have been introduced, thereby affecting the robustness of our findings. In further prospective studies based on the current model, we will consecutively expand more patients from multicenter to improve the model’s performance and extrapolation.

Conclusion

In summary, we established a clinical prediction model using demographic and clinical characteristics to assess the risk of 7-day readmission to the PICU, which may contribute to early identification and intervention for patients with a higher risk of readmission in clinical practice. It is hoped that this model will help physicians identify children who are more likely to be readmitted to the PICU and provide interventions (such as prolonging LOS) to reduce the risk of readmission and improve their long-term outcomes.