Introduction

Disability is a main problem in the aging process among the population in China and worldwide1. Disability refers to limitation or loss of an individual’s ability to perform normal activities in daily life, and increases the risk of adverse events such as frailty, falls in the elderly2. Disability not only reduces quality of life in older adults, but it also places stress on family caregivers and a burden on the health care system3. According to national data, as of January 2020, there are more than 42 million disabled elderly aged 60 and over in China, and the number is expected to reach 77 million by 2030 4. Disability is dynamic and progressive. Its occurrence and development are affected by many factors. Therefore, accurately predicting disability risk and developing targeted interventions are conducive to delaying or reducing the adverse effects of disability on quality of life.

To develop an accurate disability prediction model for the elderly, it is essential to first analyze the factors that influence disability. Previous studies have identified several such factors, including age5, depression6, smoking history7, self-rated health8, sleep duration9, drinking history10, hypertension11 and so on, but the influence of these factors on disability is not completely clear. Most scholars in China and abroad have used logistic regression and Cox regression methods to construct disability risk prediction models for older adults12,13. For instance, Chen et al. gathered data from 1,591 elderly individuals in Japan and employed the Cox regression method to construct a model for predicting the risk of disability14. The results showed that the area under the receiver operating characteristic (ROC) curve (AUC) of this model was 0.787, and its risk prediction effect was average. Moreover, confounding factors were not excluded in this study, so the applicability of the study findings needs to be verified. At present, there are some studies that use machine learning (ML) algorithm to build a disability prediction model for the elderly15,16,17, but these studies have limitations. Wu17 et al. utilized the CHARLS database and employed a machine learning approach (RF) to analyze predictors of disability among Chinese adults aged 65 and above. The results revealed that walking speed, age, and peak expiratory flow were the most significant predictors. However, this study was restricted to the RF model alone, lacking methodological diversity. In this study, we use five ML algorithms to build a disability prediction model for the elderly, and selects the elderly who have not been disabled at baseline.

ML can reflect the nature of high-dimensional data through supervised, unsupervised, or semi-supervised methods. ML can not only improve the understanding of data but can also be used to analyze high-dimensional, large, and complex relational data. With advances in computing technology during the 21st century, ML has become widely used in the field of health care to help improve the level of disease diagnosis18,19,20,21. However, ML models are often black boxes, and the visualization of intermediate processes cannot be achieved22. Therefore, in this study, the Shapley Additive Explanations (SHAP) and ML algorithms were combined to construct a visual prediction model to explain the individual prediction of kernel-based method23.

Therefore, the purpose of this study was to establish an accurate and intelligent disability risk assessment and prediction model to provide a new means for the early diagnosis and scientifically based prevention of disability among older adults and help reduce the burden on society owing to disability.

Materials and methods

Study design and participants

The data for this study were from the China Health and Retirement Longitudinal Study (CHARLS). The CHARLS is a nationally representative longitudinal database of people aged 45 years and over in China24. The study began a national survey in 2011, covering 150 counties and 450 village/neighborhood committees in 28 provinces of China. The researchers followed participants in 2013, 2015, 2018, and 2020 using face-to-face computer-assisted personal interviewing and gathered extensive information about socioeconomic status, health, and anthropometric and laboratory measurements. The study received ethics approval from the Biomedical Ethics Review Committee of Peking University, Beijing (IRB00001052-11015), and all participants provided informed consent. All study methods were carried out based on the Declaration of Helsinki. All methods were performed in accordance with the relevant guidelines and regulations.

This study used 2018 as the baseline and conducted follow-up in 2020. A total of 19,816 participants were included in the baseline survey, excluding participants aged < 65 years (12,133), those with disability at baseline (2,298), and those with missing or abnormal key predictors (2,213); ultimately, 3,172 participants were included in the analysis.

Disability assessment

The internationally accepted Katz scale was used to evaluate activities of daily living (ADL) of older adults25. ADL refers to a person’s ability to independently complete necessary activities in daily life, reflecting the most basic ability of self-care26. The scale included six items, namely, dressing, bathing, eating, getting in and out of bed, using the bathroom, and controlling urination and defecation. There were four options for each item: (1) no difficulty; (2) difficult but still possible to carry out; (3) difficult and needing help; and (4) unable to carry out. With any one item selected among items (2), (3), or (4), the individual is judged to have disability14. Cronbach’s α coefficient of the scale in this study was 0.867.

Data collection and participant variables

According to the relevant literature in China and abroad27,28,29,30, Predictors in this study included sociodemographic factors, chronic diseases, health status indicators and lifestyle factors.

Among sociodemographic factors, age was divided into age groups 65–74 years, 75–84 years, and 85 years or above. Marital status comprised married and other, including separated, divorced, widowed, or never married. Education level was categorized as below primary school, primary school, secondary school, or high school and above. The area of residence was divided into rural and urban areas. Among chronic diseases, hypertension, lung disease, arthritis, heart disease, stroke, and diabetes were assessed as presence or absence of each disease. For health status indexes, sleep time was divided into more than 9 h, 6–9 h, and less than 6–9 h. Self-rated health status was divided into very good, good, average, poor, and very poor. Depression was categorized as depression and no depression. Among lifestyle factors, drinking frequency was divided into non-drinking and drinking, smoking frequency into non-smoking and smoking, and exercise into exercising or not exercising every week.

In this study, the 10-item Center of Epidemiologic Studies Depression Scale (CES-D-10) was used to evaluate depression status among participants. The scale comprises 10 items, with a total score of 30. With scores ≥ 10, the patient was classified as having depression; with scores < 10, the patient was classified as not having depression31.

Derivation and evaluation of prediction models

In this study, the dataset was partitioned using nested cross-validation, in which a super-parametric search is performed by estimating the generalization error of the underlying model to obtain the optimal parameters of the model. Specifically, two loops—the outer loop and inner loop—are included in the nested cross-validation run. The inner layer uses the grid search method to obtain the optimal super-parameters of various models. The outer loop provides 80% of the data as the training set to the inner loop for training while retaining 20% of the data as the test set for testing the inner loop model. In this way, information leakage of data can be prevented to obtain relatively low model scoring deviation.

In this study, we conducted an in-depth investigation of disability in the fields of medicine and nursing, and we pre-screened 18 characteristic variables that are highly related to disability. During our research, we used a least absolute shrinkage and selection operator (Lasso) model to compare the effects, selected 10 additional important characteristic variables, and compared them with the total variable model to verify their functionality. Given the number of feature variables, the correlation, and the size of the training dataset, feature selection was not necessary with sufficient computational resources. Therefore, we did not take feature engineering as the main research focus for the construction of an ML model; we found no serious over-fitting phenomenon in the model.

In this study, nine parameters were used to evaluate the performance of the prediction model, such as accuracy, precision, recall, F1 Score, Hamming loss, Jaccard score, Cohen’s kappa score, Confusion matrix and ROC curve and area under curve (AUC). In addition, decision curve analysis (DCA) reflects the prediction performance of the model under different threshold probabilities. All positive and negative lines represent the extreme case of net gain when all samples are positive and negative, respectively. The flowchart for model derivation and verification is shown in Fig. 1.

Fig. 1
figure 1

Flowchart of model derivation and validation.

Full size image

ML interpretation

As a visualization tool, SHAP can explain ML models by combining the importance of predictors with predictive effects to obtain the relative risk scores of various predictors and then estimating the contribution of each predicted final prediction result from clinical data to predict the probability of the possible occurrence of various clinical events. Because SHAP can be used to quantify the related factors through calculation to improve the accuracy of the prediction results, the performance of the visual prediction model using SHAP is of great importance in improving the accuracy of clinical diagnosis and treatment32.

Statistical analysis

Continuous variables are presented as the mean ± standard deviation, and categorical variables are presented as frequency and percentage. All the above analyses were conducted using IBM SPSS 27.0. Feature selection, model derivation, and model evaluation were performed with Python 3.7.6. A two-sided p-value of < 0.05 was considered statistically significant. Five distinct machine learning techniques, including logistic regression (LR), deep neural network (DNN), support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost), were chosen to construct models.

Results

Baseline characteristics of the study population

A total of 3,172 participants were enrolled in the study; 695 participants were disabled by the end of follow-up. Sample balancing was performed based on the oversampling method. The sample distribution characteristics of the training and test sets were the same, which indicated that the grouping of the training and verification sets was completely random, and deviation caused by an uneven distribution was avoided. In the comparison of baseline characteristics, there were significant differences in age, marital status, education level, household registration, hypertension, chronic lung disease, arthritis, heart disease, stroke, diabetes, sleep time, depressive symptoms, self-rated health status, smoking, alcohol consumption, and physical exercise. The baseline characteristics of participants are presented in Table 1.

Table 1 Selected baseline characteristics of participants.
Full size table

Performance evaluation of prediction models

Results regarding the performance of the predictive model are shown in Table 2. Among comparisons of the five ML algorithms, the prediction model constructed using RF had the best performance. The F1 score and accuracy rate of the model were approximately 0.92; the recall was 0.96; the accuracy rate was 0.89; the Hamming distance was 0.08; the Jaccard coefficient was 0.86; and the kappa coefficient was 0.84. The model constructed using the XGBoost method had good performance, with an F1 score of 0.86, recall of 0.93, and accuracy rate of 0.85. The models constructed using SVM, DNN and logistic regression had average performance. After adding lasso algorithms for feature selection, the F1 scores, recalls, and accuracy rates of the RF and XGBoost models were 0.81 and 0.89; 0.84 and 0.85; and 0.80 and 0.77, respectively. The obtained performance results were relatively poor. This may be owing to the loss of some information during feature selection. The learning curve results showed that there was no serious overfitting of the model, so the results of subsequent presentations were based on the model without LASSO feature selection.

Table 2 Performance evaluation of prediction models.
Full size table

The results of the DCA of the models are shown in Fig. 2A. In general, the performance of several models was consistent with the results in Table 2. It is noteworthy that most models showed a decrease in net benefit with increased threshold value; only the RF model always showed a high net benefit. Based on the observation of its prediction probability results, the model exhibited a very high probability value for the prediction of true positive samples. When the threshold probability was increased to 0.9, the number of false positives was only 1, indicating that RF yielded the optimal clinical value in predicting the normal and abnormal categories. The AUC is shown in Fig. 2B. The results show that the RF model has the highest area under curve (AUC = 0.920). The results of the confusion matrix are shown in Fig. 3. To sum up, these results demonstrate that RF is the optimal model to use.

Fig. 2
figure 2

DCA and AUC for the prediction models.

Full size image
Fig. 3
figure 3

Confusion matrix. (A): Confusion Matrix of LR; (B): Confusion Matrix of SVM; (C): Confusion Matrix of DNN; (D): Confusion Matrix of RF. (E): Confusion Matrix of XGBoost.

Full size image

Interpretability analysis of prediction models

The prediction models constructed using RF, XGBoost, and DNN were visualized using SHAP. The predictors are shown in Fig. 4. The results showed that self-rated health was the most significant predictor of the predictive models constructed using RF. Additionally, sleep duration, educational level, arthritis status, and depressive symptoms all demonstrate high importance with similar rankings in both the Permutation Importance (PI) and SHAP analyses. From the perspective of SHAP analysis, the key influencing factors include self-rated health, education level, sleep duration, drinking history, depressive symptoms, hypertension, and arthritis. Although the overall importance distribution trend is consistent, some differences were observed in the ranking of features with moderate to low importance. For instance, the Age variable ranks last in the PI analysis of RF, whereas chronic condition ranks last in SHAP analyses of RF. While age is a known risk factor for disability, its marginal impact on the model’s prediction accuracy on the test set is relatively small when the age feature is permuted alone, given that the model can already leverage other strongly correlated variables (such as self-rated health state, arthritis status, etc.) for prediction. Conversely, chronic condition ranks lowest in SHAP, suggesting a smaller role in training fit or average contribution to the prediction magnitude. However, its higher ranking than Age in Permutation Importance implies that removing the information from chronic condition actually impairs the model’s predictive performance on the test set slightly more than removing age information. This suggests that chronic condition might retain some unique predictive value for generalization that cannot be fully substituted by age. In the prediction model constructed by XgBoost, self-rated health is the most important predictor, followed by education level and sleep duration. Self-rated health remained the most significant predictor in the DNN-constructed predictive models, followed by arthritis and depressive symptoms. In conclusion, there is a high correlation between self-rated health, education level, sleep duration, drinking history, depressive symptoms, hypertension, and arthritis, and the 2-year disability prediction probability in healthy older adults.

Fig. 4
figure 4

Relative feature importance of RF (A–C), XGBoost (D–E), and DNN (F–G). Permutation importance of RF; (B). SHAP Value of RF; (C). SHAP summary of RF; (D). Characteristic importance of XGBoost; (E). SHAP summary of XGBoost; (F). Characteristic importance of DNN; (G). SHAP summary of DNN.

Full size image

Discussion

In this study, we established a model to predict the 2-year risk of disability risk in older adults aged 65 and above in China who had good health status. We applied five machine learning algorithms to construct the prediction model. Among these models, the random forest model exhibited the best predictive performance. By using SHAP to analyze the best-performing models, we identified key influencing factors, including self-rated health, education level, sleep duration, drinking history, depressive symptoms, hypertension, and arthritis.

We found that self-rated health status was most effective in predicting the disability risk in older adults. Individuals with poor self-rated health often have negative perceptions of their own well-being and may neglect proper health management, which can, in turn, exacerbate disability33. Low educational level also increased the risk of disability among older adults34. Individuals with low education levels often lack knowledge about health, disease prevention, and chronic condition management, which can increase their risk of disability35. Sleep duration is another risk factor for disability in older adults. Chronic sleep deprivation may lead to health issues such as a weakened immune system, metabolic disorders, cardiovascular disease, and hyperglycemia. The onset of these chronic diseases may elevate the risk of disability9. Excessive sleep duration often means reduced daily activity, which can lead to issues such as muscle atrophy and osteoporosis, thereby increasing the risk of disability36. We found that depression may increase the risk of disability in older adults. This finding is consistent with the results of previous studies37,38,39. Physiologically, depressive symptoms may exacerbate the inflammatory process, thereby increasing the risk of disability. In addition, depression may cause physical fatigue and pain in the elderly, leading to a decline in their ability to perform daily activities40. Psychologically, depression in the elderly may lead to reduced medication adherence41. We also found that among six chronic diseases, older adults with hypertension or arthritis were at higher risk of disability.

The systematic review42 indicated that while existing prediction models for disability in older adults demonstrate acceptable discrimination, their overall quality and clinical value remain substantially limited. We acknowledge that current research still exhibits limitations. Based on the findings of the systematic review, our work is somewhat improved. First, data imbalance can affect the performance of the prediction model43. Among the subjects, there is an imbalance between disabled and non-disabled elderly individuals. To address this issue, this study employs the oversampling method to mitigate data imbalance, thereby enhancing the prediction accuracy and stability of the model. Regarding data processing, we rigorously excluded samples with pre-existing functional disability at baseline. Secondly, AUC is the most commonly used evaluation metric. In a study that used the same database as this one to predict disability in the elderly, the researchers constructed six models with AUC values ranging from 0.790 to 0.833 44. In this study, the AUC of the five machine learning algorithms ranges from 0.664 to 0.920, suggesting that our research can serve as a valuable reference for enhancing disability prediction models for the elderly in China. Additionally, we employed SHAP analysis to improve model interpretability and identify key predictive factors. In the present study, the model performance was evaluated using nested cross-validation and nine parameters, including accuracy, precision, and recall. In addition to the conventional indicators, other indicators were included to comprehensively verify the performance of the model. We found that ensemble learning methods (such as RF and XGBoost) performed best among the five models, which may be because these form a stronger classifier by combining multiple weak learners; thus, if some weak learners make prediction errors, other weak learners can correct the errors to some extent45. In summary, by addressing issues such as data imbalance, variable selection, and comprehensive evaluation, we have developed a highly effective disability prediction model for the elderly in China.

There are some limitations in this study. First, the sample size is limited because some subjects are excluded due to missing variables, which may affect the generalization of the model. Second, we can only predict the risk of disability in the elderly two years later. Future studies will need to expand the sample size and increase the prediction period. Thirdly, to improve the universality of the prediction model, external verification should be performed to evaluate the generalization of the model. Finally, in the predictive model in this study, we only considered two states of ADL; disability should be divided into mild, moderate, and severe disability in future studies. Future work should focus on multi-center verification, increasing sample size, and prolonging the time span. This will help to take a substantial step in translating the predictive model from theory to clinical application.

Conclusion

Disability significantly impacts the quality of care and well-being of older adults, imposing a substantial social burden. The results of this study suggest that the use of high-performance ML models for disability risk detection is recommended and has the potential for clinical practice. The ML models developed using RF and XGBoost algorithms demonstrated robust predictive performance and a certain degree of generalizability. These models can effectively identify older adults at high risk of disability, enabling targeted interventions.

ML models offer potential for improving the assessment of disability risk and designing targeted prevention strategies. To mitigate disability risk and promote healthy aging, healthcare professionals can pay more attention to older individuals with low self-rated health scores, low educational levels, short sleep duration, regular alcohol consumption, depressive symptoms, and hypertension or arthritis. Tailored interventions, such as promoting healthy lifestyles and managing blood pressure, might play a pivotal role in preventing disability in these high-risk populations. Future research should focus on external validation using multicenter data to further enhance the applicability and reliability of these models.