Introduction

Mechanical ventilation (MV) is often used to treat acute or chronic respiratory insufficiency, as well as respiratory failure caused by various factors, in order to save the lives of critically ill patients1. Approximately 50% to 60% of patients in the ICU require MV to improve their oxygenation. However, as the duration of MV increases, so does the incidence of complications associated with ventilators, such as ventilator-associated pneumonia, respiratory muscular dystrophy, pulmonary fibrosis, and pneumothorax2. This increase in complications, coupled with longer ventilation times and offline periods, leads to a vicious cycle that results in higher morbidity, mortality, and hospitalization expenses2. Therefore, once the patient's primary disease is under control, their respiratory function is corrected, and their ventilation and oxygenation are stable, it is important to consider removing the ventilator as soon as possible.

According to a consensus reached by the European Respiratory Association (ERS), American Thoracic Association (ATS), European Intensive Care Association (ESICM), and Emergency Medicine Association (SCCM) in 2007, a spontaneous breathing test (SBT) is used to assess the feasibility and classification of offline periods. The classifications of SBT include Simple weaning, Difficult weaning, and extended offline3. In the ICU, SBT is widely used to evaluate whether patients are suitable for being taken off the ventilator4. The standard for successful weaning is that MV is not required again within 48 h after extubation5. Difficulty offline is a significant challenge in clinical practice. Reviewing the literature reveals that only half of the patients who have been on MV for more than 12 h can successfully is take off the ventilator, while nearly 40% will encounter difficulties in the weaning process6,7,8. This not only causes great suffering for patients, but also increases the social and economic burden and consumes a significant amount of medical resources. Therefore, a more accurate prediction model is needed to anticipate the success of weaning in patients.

Several studies have explored the prediction of successful extubation using different approaches. Kuo et al.9 used an ANN model, which was accurate but computationally complex. Fabregat et al.10 employed logistic regression, gradient boosting, and support vector machine algorithms, offering various options but requiring more data and tuning. Meng et al.11 proposed an ANN for difficult weaning, but its complexity limits clinical application. Liao et al.12 reduced ventilator duration using a machine learning algorithm, but its performance for unsuccessful weaning is unclear. Liu et al.13 used a decision tree model for extubation outcomes, a simple but potentially overfitting approach. Zeng et al.14 introduced an RNN model for dynamic risk assessment, adaptable but computationally demanding. It is worth noting that most of these studies were initiated by doctors, a large number of parameters for prediction are used, which are difficult to collect in clinical practice. An important role of nurses in intensive care unit is to diagnose whether patients are ready to remove the ventilator. Nurses can quickly and appropriately reduce the risk of using ventilator by reducing ventilation protection, so as to withdraw the ventilator in time. Therefore, the purpose of this study is to develop a simple artificial intelligence (AI) model from the perspective of nurses. This model uses several parameters and determines the simplest key predictors, so that the model can achieve enough accuracy with as few parameters as possible to predict weaning time.

In summary, this study comprehensive consideration of nurse-centered perspective, simplified and effective prediction model design, personalized treatment scheme and early intervention measures, which offers more accurate predictions and guidance for the weaning process of ICU patients with mechanical ventilation. This study can assist nurses in systematically evalu-ating patients before the SBT test and choosing the appropriate test timing. Moreover, it helps nurses to identify patients with a difficult weaning tendency and provide early intervention and preventive measures. Based on the scoring system’s results, nurses can tailor personalized nursing plans, including respiratory therapy, rehabilitation training, and nutritional support, to improve patients’ spontaneous breathing ability and the rate of successful weaning.Its provides brand-new methods and guidance for the weaning process of patients with mechanical ventilation and is expected to have a positive impact in clinical practice.

Materials and methods

Study participants

A total of 564 patients who met the study inclusion criteria were collected from the third-class first-class general hospital. Among them, 37 patients to transfer to another hospital, 23 patients discontinued treatment and left the hospital, and 17 patients experienced unplanned extubation during MV. Based on the offline outcome, these 487 patients were divided into two groups: 323 patients (66.32%) were simple weaning, and 164 patients (33.68%) faced difficult weaning. This study was approved by the hospital ethics committee (number: 2022-051).

In this study, we conducted a retrospective cohort study to select patients who underwent invasive MV in the ICU of the third-class first-class general from June 2018 to December 2022. Inclusion criteria were as follows: (i) age ≥18 years old, (ii) patients with tracheal intubation and ventilator-assisted ventilation, (iii) the primary disease was under control and a SBT test was performed based on the doctor's advice, with the plan to remove the tracheal intubation, (iv) complete preservation of case data. Exclusion criteria were as follows: (i) patients with pre-intubation airway dysfunction that could not be improved in a short time, such as airway compression due to tumor, congenital airway malformation, airway burn, or airway surgery, (ii) long-term MV exceeding 21 days, (iii) incomplete data.

Data collection

The study utilized the following information: (i) Demographic: age, sex, consciousness, cough reflex intensity15 (0 = no cough response, 1 = audible movement of air through the endotracheal tube but no audible cough, 2 = strong cough with phlegm under the end of endotracheal tube, 3 = strong cough with phlegm coming out of the end of endotracheal tube) ,body mass index (BMI), nutritional risk screening (NRS-2002), acute physiology and chronic health status score II (APAHE II), sequential organ failure score (SOFA), (ii) Questionnaire on related factors: duration of MV, vasoactive drugs on the day of weaning, last day liquid balance, temperature at SBT, heart rate(HR) at SBT, blood pressure at (BP)SBT, respiratory rate(RR) at SBT, (iii) Blood gas analysis parameters: PH value, PaO2, PaCO2, PaO2/FiO2, serum potassium, serum potassium, base excess (Be), LAC (lactic acid), (iv) ventilator parameters: tidal volume, PS, PEEP, (v) Laboratory inspection index: hemoglobin, hematocrit, platelet count, white blood cell count, high-sensitivity C-reactive protein, PCT procalcitonin, total protein, albumin and globulin. A total of 36 variables were included in the analysis.

Machine learning models

With the advancement of computing power and the availability of high-frequency medical data, artificial intelligence (AI) methods based on machine learning (ML) have been widely adopted in medical decision-making. By analyzing large amounts of medical data, including clinical records, medical images, and genomics data, machine learning models can learn patient-specific patterns and accurately predict future events16,17. Currently, commonly used ML algorithms include logistic regression (LR), random forest (RF), support vector machines (SVM), light gradient boosting machine (Light GBM), and XG Boost. LR18,19,20,21 is a regression algorithm for solving classification problems and uses the sigmoid function to map linear regression model outputs to probabilities between 0 and 1. RF22 is an ensemble classification and regression algorithm that uses decision trees. SVM23 creates a binary classifier and is used in small sample data and binary classification. Light GBM24 and XG Boost25 both use gradient boosting with decision trees, but Light GBM features exclusive feature bundling and gradient-based one-sided sampling, while XG Boost uses regularization and column sampling to prevent overfitting and improve efficiency.

Statistical analysis

The SPSS 26.0 software was used for statistical analysis in this study. Categorical variables were represented as number and percentage and were compared using the Chi-square test, Fisher’s exact test, and non-parametric Mann–Whitney U test. Continuous variables were compared using t-test and non-parametric Mann–Whitney U test.

The predictive model was developed and validated using the Python 3.9 language platform. The dataset was randomly divided into a 7:3 ratio for training and verification sets, the model is trained on the training set, and the generalization ability of the model is evaluated on the test set. Five models, namely Logistic, SVM, Random Forest, XGBOOST, light GBM, are used to model and predict, and the hyperparameter of the model is selected by grid search and 50% cross-validation. Reach the confusion matrix models. According to the confusion matrix, the precision rate, recall rate, accuracy rate and F1 value of the model, respectively. Draw the ROC curves of the five models. Compare ROC curves of five models. Select the best model among the five models.

Results

Baseline characteristics

According to the inclusion criteria outlined in Fig. 1, In this study, the data of 341 (70%) offline patients were used as the training set to build the model, and the data of 146 (30%) offline patients were used as the verification set. Among the training set, 70 patients (20.5%) experienced offline difficulties during the offline process. In the verification group, 33 patients (22.6%) encountered offline difficulties. The distribution of each characteristic variable in the training set and the verification set is statistically different.

Fig. 1
figure 1

Flow chart of selection of participants.

Full size image

As shown in Tables 1 and 2, the cough reflex intensity, consciousness, tidal volume, BMI, PaO2, PaO2/FiO2, total protein, white blood cell count(WBC), globulin, PH, C-reactive protein(CRP) SOFA, Respiratory rate(RR),hemoglobin, Be, APACHE II, GCS, and Duration of MV were statistically significant (P < 0.05).

Table 1 Single factor analysis of classified variables.
Full size table
Table 2 Single factor analysis of continuous variables.
Full size table

Model development

The dataset is split into a 7:3 ratio for training and testing. Five models Logistic, SVM, Random Forest, XG boost, and Light GBM are trained on the training set and evaluated on the test set to assess their generalization ability. Hyperparameters for each model are selected using grid search and 5-fold cross-validation. The confusion matrix for each model is presented in Fig. 2.

Fig. 2
figure 2

Confusion matrix for each of the five machine learning methods [(i) Logistic, (ii) SVM, (iii) Random Forest, (iv) XGBOOST, (v) light GBM for predicting patient weaning. ROC-AUC: Receiver operating characteristic curve area under the curve.

Full size image

The sensitivity rate, recall rate, accuracy rate, and F1 value of the model are calculated based on the confusion matrix, as presented in Table 3. The results indicate that the Random forest model exhibits the highest performance. It shows that the Random forest model outperforms the other ML models. Random forest in terms of accuracy (0.748), recall (0.888), specificity (0.767) and F1score (0.825). The Random forest model showed high accuracy, with AUC of 0.805, while the XG Boost displayed relatively similar high accuracy, with AUC of 0.800, respectively (as illustrated in Fig. 3).

Table 3 Performance comparisons of five machine learning methods.
Full size table
Fig. 3
figure 3

Receiver operating characteristic curves for each of the five machine learning methods. ROC-AUC of the five models [(i) Logistic, (ii) SVM, (iii) Random Forest, (iv) XGBOOST, (v) light GBM for predicting patient weaning. ROC-AUC: Receiver operating characteristic curve area under the curve.

Full size image

Feature importance analysis

Random Forest model reduced risk of overfitting flexibility can handle both regression and classification. Can determine feature importance. Predictor importance was calculated with the Random Forest model. The SHAP values for the highest predictors are shown in Fig. 4. The SHAP value diagram displays each sample as a point, with the x-axis representing the SHAP value of the feature and the y-axis representing each feature. The color of the point reflects the magnitude of the feature value: red for positive impact and blue for negative impact. For instance, considering the number of days of catheterization, a smaller value corresponds to a positive SHAP value, suggesting that a shorter duration makes extubation easier, especially when the patient is conscious. The most important predictive feature of extubation failure was Duration of MV. The majority of important predictors can be grouped into Consciousness, APACHE II, RR at SBT, GCS, APACHE II and hemoglobin.

Fig. 4
figure 4

SHAP values for most important predictors of extubation failure. Overview of SHAP values for the top predictors of difficult weaning (negative SHAP values) or simple weaning (positive SHAP values). Features are ordered according to importance.

Full size image

In this study, we conducted calibration curve plotting to evaluate the performance of various models. We compared the Random Forest model with Logistic, SVM, XGBOOST and Light GBM models. Figure 5 illustrates that the prediction probability of the Random Forest model. The Random Forest model exhibited superior calibration compared to the other models, both in the test and verification set datasets. This further confirms the effectiveness of the Random Forest model in accurately predicting outcomes.

Fig. 5
figure 5

Calibration curves of Logistic, Random Forest, SVM, Light GBM, XGBoost.

Full size image

Discussion

Machine learning technology has shown great potential in the field of healthcare management26. However, each model has its limitations. For instance, LR tends to overfit due to the use of numerous low-significance variables to enhance accuracy. Similarly, SVM may not be robust enough when analyzing large datasets.

Most clinicians lack expertise in machine learning, making it challenging to develop models. Additionally, machine learning algorithm experts often struggle to translate model results into clinical recommendations. To address this issue, we compared the performance of five machine learning algorithms in predicting weaning difficulties in ICU patients on MV and selected the best method based on prediction accuracy. When comparing the average AUC value of Random Forest with other models through five-fold cross-validation, it is observed that the AUC value of Random Forest remains stable, providing a more objective evaluation of the model performance. This stability is not influenced by data instability or particularities, making it a more comprehensive method to assess the model's generalization ability and stability.

Tracheal intubation is a crucial technique for ensuring effective ventilation in the ICU. However, extubation carries a risk of high failure rates and serious complications, making offline assessment an essential part of treatment27. Foreign studies28,29 have shown that timely and continuous offline evaluation, compared to doctors and nurses, reduces MV time, hospitalization duration, and the risk of complications while improving patient rehabilitation and survival rates. By utilizing computer algorithms, we can predict the risk of offline difficulty in ICU patients on MV accurately.

This study demonstrated that the likelihood of simple weaning increases. Silva-Cruz et al.30 confirmed that shorter MV durations lead to higher extubation success rates. Numerous studies have indicated that long-term MV can result in disuse atrophy of respiratory muscles31,32,33. After being taken off the ventilator, respiratory muscles tend to lose their function due to a lack of exercise, making long-term spontaneous breathing challenging. This respiratory factor contributes to weaning failure and increases the risk of ventilator-associated pneumonia34. Additionally, the longer the duration of MV, the greater the dependence of the patient on the ventilator35. Beyong et al.36 reported a ventilator dependence incidence of 13.68%, which further increases the difficulty of weaning.

Studies have shown that consciousness, RR of SBT, GCS, APACHE II and hemoglobin are considered to be very important characteristics. It is found that the probability of successful extubation with GCS score ≤ 8 is 33%37. The state of consciousness is closely related to whether the patients can extubate38. The lower GCS score, the more serious the disturbance of consciousness, which leads to the obvious inhibition of cough reflex and respiratory center, which leads to the prolonged mechanical ventilation time and the difficulty of ventilator offline. Medical staff should strengthen the airway management of patients with consciousness disorder based on GCS score39 and take targeted measures in advance to reduce the incidence of offline difficulties. APACHE II can quickly and accurately evaluate the disease risk of ICU patients40. The greater the APACHE II score, the more critical the patient's condition becomes, thereby increasing the likelihood of encountering challenges during the process of discontinuing critical care or ventilator support41. The respiratory frequency during SBT test is an important index, which can reflect the respiratory load and respiratory muscle fatigue of patients. It is generally believed that the respiratory rate during SBT test should be less than 35 beats/minute, otherwise it may suggest that patients are unable to tolerate SBT or the risk of failure in weaning42. The change of hemoglobin concentration is related to the pathophysiological changes of patients during offline, hemoglobin plays a critical role in oxygen delivery and may affect cardiac workload, work of breathing and respiratory muscle endurance43. Lai et al.44 found that patients with Hb value higher than 80 g/L are more likely to be offline successfully than patients with Hb value lower than this threshold.

A number of studies45,46,47 have shown that the treatment of weaning from ventilator under the guidance of nurses in charge is not only safe and effective, but also produces weaning from ventilator faster than the traditional treatment under the guidance of doctors. Nurses play a crucial role in closely monitoring the patient's level of consciousness and assessing nervous system function using the Glasgow Coma Score. Detecting and addressing neurological issues promptly is essential for facilitating the weaning process. Additionally, conducting a comprehensive evaluation of the patient's APACHE II score and hemoglobin levels can serve as a foundation for developing a personalized weaning plan and adjusting airway management strategies. For instance, implementing airway clearance techniques for patients experiencing difficulty in weaning can enhance oxygenation levels and reduce mechanical ventilation duration. Monitoring hemoglobin levels and promptly addressing any anemia or blood-related issues is vital for ensuring adequate oxygen supply and tissue perfusion during the weaning process. Moreover, adjusting mechanical ventilation settings, initiating early mobilization or physical therapy, and optimizing nutritional support are key components in facilitating a successful weaning process. Monitoring the patient's respiratory rate during Spontaneous Breathing Trials (SBT) and making necessary adjustments to respiratory support levels can effectively contribute to the seamless progression of the weaning process. Nurses assess the risk of patients leaving the ventilator and adjust their strategies for leaving the ventilator. Strengthening airway management of patients with mechanical ventilation and enhancing ventilator function training can effectively and safely prepare patients for weaning and reduce the occurrence of difficult weaning.

We have revised Nurses in ICU can use the findings of this study to help patients and their families understand the importance and process of ventilator weaning through education and communication. They can make or adjust the nursing plan according to the personalized prediction model provided by the research to ensure that the withdrawal process is safe and effective. Using the model for continuous monitoring and evaluation, nurses can identify possible offline difficulties early and take preventive measures. In addition, nurses can use the predicted results to support clinical decision-making and provide detailed information about patients' condition to doctors. Through the explanation of scientific basis, nurses can enhance patients' confidence in the process of withdrawal and reduce their fear and anxiety. At the same time, nurses encourage patients to participate in the withdrawal process, adjust nursing plans according to patients' feedback, provide psychological support, record key data, and continuously update their professional knowledge, so as to achieve the best patient care and promote patients' rapid recovery.

To address these limitations and expand the applicability of our findings, future research should aim to increase sample size, enhance the statistical power of the study, and reduce potential bias. Conducting multicenter or cross-regional studies would help validate our results and determine their relevance in a broader context, reducing the bias inherent in single-center studies. Additionally, ensuring a more diverse sample population—including different ages, genders, races, and socioeconomic backgrounds—can increase the generalizability of study results and reduce bias toward specific groups.

Conclusion

This study underscores the potential application of machine learning in predicting difficult weaning among ICU patients. Nurses can use this model to quickly evaluate patients' offline preparation, accurately predict which patients may face offline difficulties, and prepare corresponding nursing measures in advance. Patients may experience fewer complications and faster recovery process, because the model helps to remove the ventilator in time and accurately. Hospitals can reduce the cost of treatment and improve the efficiency of resource utilization by reducing the use time of ventilators and the incidence of complications. In addition, accurate prediction helps to improve medical quality, promote patient safety, support clinical decision-making, and promote data-driven medical practice. In the end, this study not only improved the quality of patient care, but also improved the efficiency and effectiveness of the entire medical system.

Hence, future research endeavors should prioritize the validation of these models in clinical settings, enabling medical practitioners to make evidence-based decisions grounded in data. The ultimate aim is to enhance patient outcomes and mitigate healthcare costs.