Introduction

Rehabilitation is a critical component of universal health coverage and is no longer limited to a small segment of the global population1. The aging global population, coupled with the increasing prevalence of chronic diseases, disabilities, and severe illnesses, has led to a growing demand for rehabilitation services2,3. Recent data indicated a significant rise in rehabilitation needs, showing an estimated 63% increase in the number of people eligible for such services between 1990 and 2019. Moreover, projections suggest that by 2030, up to 241 million people worldwide will require rehabilitation3,4. However, existing rehabilitation resources remain insufficient, particularly in low- and middle-income countries, where more than half of the population lacks access to essential rehabilitation services5,6,7,8. Furthermore, rehabilitation services are primarily concentrated in secondary and tertiary healthcare institutions, even though most common and chronic conditions could be effectively managed at the primary healthcare level9. Therefore, optimizing the allocation of rehabilitation resources is essential to ensure broader access to timely, safe, equitable, and effective rehabilitation services for all.

Compared to other countries, China faces an especially high demand for rehabilitation, with an estimated 460 million rehabilitation visits reported in The Lancet’s Global Burden of Disease published in December 20204. In contrast to the well-established tiered rehabilitation systems found in the United States and several European nations, China’s rehabilitation service system remains underdeveloped10. To address the imbalance between demand and supply, China introduced a three-tiered rehabilitation care system in 2012 and has made considerable progress in recent years in reducing the disparities between rehabilitation needs and available resources11,12. However, the distribution of services remains suboptimal13; many individuals still experience delays in obtaining rehabilitation, and issues of accessibility and equity persist in service delivery12.

Clinicians play a central role in hierarchical healthcare systems, making critical decisions that directly affect rehabilitation outcomes. These decisions are influenced by various factors, including clinical experience, patient preferences, family involvement, and health insurance policies14. As a result, clinicians’ assessments might vary, potentially leading to delays or missed opportunities for timely rehabilitation intervention. In certain regions of China, particularly in the southwest, there is a notable shortage of rehabilitation experts, with general practitioners (GPs) increasingly taking on roles traditionally filled by experts15. In these settings, primary care physicians are often responsible for determining whether patients require rehabilitation, the type of rehabilitation they should receive, and for referring them to appropriate services. However, many primary care providers lack the specialized knowledge to accurately assess a patient’s rehabilitation needs. Furthermore, limited studies have examined the capacity of primary care physicians to identify and assess rehabilitation service needs, especially in community and rural settings16. Therefore, this study aimed to compare the assessments of rehabilitation service needs made by clinicians and experts, as well as to investigate differences across levels of healthcare institutions. This study also seeks to lay the groundwork for future research on rehabilitation service needs.

Methods

Study design

We conducted a multicenter clinical vignette-based survey from May to October 2024 in Anhui Province, located in the middle of China. This province was chosen due to its relatively underdeveloped economy, limited medical resources, and the availability of comprehensive institutional data. A total of 40 clinical vignettes, primarily representing rehabilitation cases, were selected from our previous study for evaluation by clinicians and experts17. Each clinical vignette was presented in a standardised and structured medical record format, including patient demographics, chief complaint, history of present illness, past medical history, personal history, family history, epidemiological history, and findings from physical examination. This structured information was sufficiently detailed to allow raters to independently assess each patient’s rehabilitation service needs. All vignettes followed a consistent structure to minimize information bias and ensure comparability across raters. An example of the clinical vignette is provided in Appendix 1. The inclusion criteria for clinicians were: (1) relevant clinical experience and (2) consent to participate. The inclusion criteria for experts were: (1) at least 20 years of clinical experience in rehabilitation medicine, (2) a senior professional title (e.g., Associate Chief Physician or higher), (3) consent to participate. Clinicians in China follow a standardised certification and training system. To provide the professional clinical service, all clinicians must first complete a medical degree (3–8 years in duration depending on the training model), pass the National Medical Licensing Examination (NMLE), and obtain a practicing certificate. They are also required to undergo standardised residency training, the duration which varies according to their level of prior medical education. Professional titles (junior, intermediate and senior) are awarded based on a combination of academic qualifications, years of clinical practice, and passing professional qualifying examinations.

Sample

A three-stage approach was employed to select the sample for this study. First, 40 vignettes were chosen from a database of over 2,000 medical vignettes stored in the mini-program developed in our previous study18. This mini-program was built based on a novel clinical tool developed by our research team to assess patients’ rehabilitation needs and determine appropriate service referral levels. The structure and logic of this tool are illustrated in Fig. 1. These 40 vignettes represented three different levels of healthcare institutions in Yunnan Province, including one county hospital, one township health center, and two village health centers. Second, 174 clinicians were recruited from three levels of healthcare institutions in Anhui Province, which included county hospitals, township health centers, and village health centers. After excluding 10 invalid questionnaires, 164 valid clinician responses remained, yielding an effective response rate of 94.2%. Third, three rehabilitation experts were selected to assess the clinical vignettes. The experts were internally recruited from our study team using a convenience sampling method. Eligible experts were required to have at least 20 years of clinical experience in rehabilitation medicine, hold a senior professional title, and consent to participate. Their extensive experience and expertise helped ensure the reliability and credibility of the reference standard evaluations. As a result, the final analytical sample included 167 participants: 164 clinicians and three experts.

Fig. 1
figure 1

(a) The novel rehabilitation tiered service tool for rehabilitation patients distribution in previous study. (b) The novel clinical tool for distributing rehabilitation patients in China developed by our research group. * Note: The tool was developed by our research group. (a) Was the tool, (b) was the diagram and logic path of the tool. a The Longshi-scale was an evaluation method of self-care ability among disabled people, which was approved by the National Standards Commission of China in 2018 (GB/T37103-2018). b The patient should be referred if his/her functional status did not change for over one month. c The disease course could be different in different cities. d Multiple dysfunctions indicated patients who had conscious impairments, or in addition to motor impairment, they also had any one or more cognitive impairment, speech impairment, swallowing impairment, or cardio-pulmonary impairment. This tool classifies patients based on rehabilitation needs. The ‘only movement disorders’ category refers to patients with motor impairments but no cognitive or speech disorders. These cases are considered mild and suitable for primary healthcare, as referring them to higher-level institutions would unnecessarily use up medical resources.

Full size image

Procedure

In previous studies, data were collected using a mini-program tool on WeChat, which securely stored the information in the cloud (Kanghui platform). This setup allowed for real-time access and efficient management by the researchers. The study was conducted in three phases.

Phase 1: participant inclusion and vignette screening

First, vignette data from Yunnan Province were selected from among the 28 provinces and cities included in the previous multicenter study. Vignettes were then exported from the databases of four institutions in Yunnan Province: one tertiary hospital, one secondary hospital, and two primary care institutions. 10 vignettes were randomly selected from each institution, resulting in a total of 40 vignettes for subsequent evaluation. Patient identities and addresses were anonymized to protect privacy and minimize bias. Subsequently, 174 doctors from various healthcare facilities (county hospitals, township health centers, and village health centers) in Yunnan province were invited to assess clinical vignette data developed for this study. Each doctor independently evaluated 40 vignettes presented as image-based excerpts from patient medical records. After excluding invalid responses, 164 valid assessments were included in the follow-up analysis.

Phase 2: establishing a reference standard

To establish a reference standard, two experienced rehabilitation experts were invited to independently evaluate the rehabilitation service needs of patients based on the 40 vignettes. However, 21 of the 40 vignettes displayed discrepancies between the evaluations of the two experts, and the initial inter-rater consistency among the experts was low. Therefore, a structured consensus process was implemented: another expert was invited to join the original two experts to review and discuss the vignettes with inconsistent assessments. Through systematic discussion in a closed session, the three experts reached a final consensus for each disputed case, which served as the reference standard for evaluating the clinicians’ assessments. This staged consensus approach—consisting of independent assessments followed by group discussion—is commonly adopted in rehabilitation and referral research when no objective gold standard exists19,20,21,22.

Phase 3: consistency of clinicians’ assessments of rehabilitation service needs

The 164 clinicians were asked to evaluate the rehabilitation service required for the same patient vignettes, relying solely on their own clinical experience, without considering any external factors. The clinicians’ assessments were then compared to the expert-established reference standard to evaluate their consistency. Additionally, the consistency across different levels of healthcare institutions was also exmined.

Measurement

General characteristics

This study included 164 clinicians, and their general characteristics were collected, including age, gender, years of experience, organization level, and other relevant variables. Additionally, each of the 40 patient vignettes contained standardised information such as gender, age, diagnosis, dysfunction, disease duration, and the stability of the current condition, among other details.

Clinicians’ subjective assessment outcomes

Clinicians’ subjective assessments were based on their clinical experience and the standardised information provided in the patients’ vignettes. This included patient complaints, disease progression, imaging results, type of dysfunction, and other relevant data. Clinicians were not instructed to follow any predetermined standard when deciding on the rehabilitation treatments they deemed necessary for each patient. This approach was designed to reflect the typical evaluation process they would use in routine clinical practice. The objective was to optimize clinicians’ decision-making while preserving the authenticity of real-world consultation scenarios. The assessments made by clinicians and experts were categorized into six distinct outcomes:

  1. (1)

    Outpatient rehabilitation treatment: patients’ rehabilitation needs can be met without hospitalization.

  2. (2)

    Other clinical departments: if the consultation showed no need for rehabilitation, patients might be advised to visit other clinical departments for further evaluation or treatment.

  3. (3)

    Inpatient rehabilitation in primary care: this category includes community health service centers or township health centers that provide basic rehabilitation services for patients in the recovery phase.

  4. (4)

    Inpatient rehabilitation in secondary care: this category comprises rehabilitation hospitals or departments within secondary general hospitals that offer comprehensive rehabilitation services while engaging in teaching and research.

  5. (5)

    Inpatient rehabilitation in tertiary care: these are specialized institutions that provide advanced rehabilitation services, conduct high-level scientific research, and offer educational programs for professionals, typically affiliated with regional or national hospitals.

  6. (6)

    Nursing homes or long-term care institutions: patients whose conditions extend beyond the coverage period of insurance or Medicare may be referred to care homes or nursing homes for long-term care.

Statistical analysis

The primary focus of the analysis was to assess the consistency between the clinicians’ subjective evaluations of rehabilitation service needs and the expert-established reference standard. Additionally, we examined the consistency in evaluation outcomes between rehabilitation experts and clinicians at various healthcare institutions. Descriptive statistics were used to analyze the clinicians’ general characteristics. Continuous variables were expressed as median values (IQR), while categorical variables were presented as frequencies and proportions. To assess the initial consistency among experts before consensus, Fleiss’ Kappa was calculated. Given the low initial consistency, a structured consensus process was implemented to finalize the reference standard. Furthermore, Kendall’s coefficient of concordance (W) was used to examine the overall consistency between clinicians and experts, and to compare consistency across different healthcare levels. This non-parametric statistic is appropriate for measuring consistency among multiple raters when the variables are ordinal. Given that the rehabilitation service needs in this study were categorized into six ordinal levels and each vignette was rated independently by clinicians, Kendall’s W provided a robust and interpretable measure of inter-rater reliability across different healthcare settings.

Ethic approval

The study was approved by the Ethics Review Office of Shenzhen Second People’s Hospital (Ethics No.2023-226-02PJ). All methods were performed in accordance with the relevant guidelines and regulations. All participants provided informed consent before their inclusion in this study.

Results

General characteristics of participants

Among the 164 clinicians surveyed, 92  worked in village health centers, 66 in township health centers, and 6 in county hospitals. The majority of participants were male (75.6%), with a median age of 49 years (IQR: 45–53) and a median of 28 years of clinical experience (IQR: 23–32). In term of educational qualifications, 29 participants (17.6%) held a bachelor’s degree, while the largest proportion (32.3%) had only completed vocational secondary school . Most participants (98.8%) had obtained the required professional qualifications. However,  53.9% did not hold any professional titles, while 30.9% held junior titles, 10.3% held intermediate titles, and 4.2% held senior titles. Among the 40 patient vignettes sampled, 15 (37.5%) were diagnosed with orthopedic disease, 10 (25.0%) with neurological disease, and 6 with cardiopulmonary disease, cancer, or childhood diseases (Table 1).

Table 1 General characteristics of clinicians (N = 164).
Full size table

Clinicians’ and experts’ assessments of the service needs for rehabilitation patients

The assessments made by both clinicians and experts regarding the service needs of rehabilitation patients were categorized into six distinct groups. According to the experts, the distribution of assessments was as follows: outpatient rehabilitation (50.0%), secondary inpatient rehabilitation (5.0%), primary inpatient rehabilitation (15.0%), tertiary inpatient rehabilitation (17.5%), other clinical departments (12.5%), and nursing or care homes (0%).

For clinicians, the distribution of 6,560 assessment results was as follows: outpatient rehabilitation was the most frequently selected category 43.8%, followed by primary inpatient rehabilitation (19.5%), secondary inpatient rehabilitation (17.9%), and tertiary inpatient rehabilitation (10.3%). Other clinical departments and nursing or care homes accounted for 7.3% and 1.1%, respectively. Outpatient rehabilitation emerged as the most common choice among clinicians across all levels of healthcare institutions, consistent with the overall trend. In contrast, assessments for other clinical departments and nursing or care homes constituted a small percentage. The distribution of clinician assessments across county hospitals, township health centers, and village health centers was generally consistent with the experts consensus. However, a notable difference was observed in township health centers, where more vignettes were categorized as primary inpatient rehabilitation rather than secondary inpatient rehabilitation (Table 2).

Table 2 Current situation of the service needs in rehabilitation patients of the clinicians’ and experts’ assessments.
Full size table

Consistency between the clinicians’ evaluations and the “reference standard”

Figure 2 illustrates the consistency rates between clinicians and the expert-established reference standards, along with the 95% confidence intervals for each level of healthcare institution. The overall consistency rate between the 164 clinicians and experts was 35.2%, and this rate declined progressively with lower levels of healthcare institutions, with county hospitals, township health centers, and village health centers showing consistency rates of 40.4%, 36.1%, and 34.2%, respectively. Kendall’s W consistency test further supported these results, with specific data detailed in Table 3. The overall level of consistency was low, with a Kendall’s W coefficient of 0.140 (P < 0.001) between the 164 clinicians’ assessments and the expert reference standard. This consistency declined with the level of the healthcare organization, as indicated by Kendall’s W coefficients for county hospitals, township health centers, and village health centers being 0.439 (P < 0.001), 0.209 (P < 0.001), and 0.104 (P < 0.001), respectively.

Fig. 2
figure 2

Consistency rates between clinicians’ and experts’ assessments by institution level (with 95% confidence intervals).

Full size image
Table 3 Comparison of consistency between clinicians and experts’ assessments.
Full size table

Discussion

The imbalance between the demand for rehabilitation services and available resources has become one of the critical issues that urgently needs to be addressed globally, particularly in low- and middle-income countries18,23,24. As primary decision-makers in clinical referrals, clinicians’ diagnostic and therapeutic abilities play a crucial role in the overall continuum of rehabilitative healthcare and may affect the rehabilitation services received by patients. Therefore, the purpose of this study was to investigate the ability of physicians in low-level healthcare organizations to assess patients’ needs for rehabilitation services. We analyzed data from a cross-sectional, multicenter study conducted in China, focusing on the current state of clinicians’ assessments regarding rehabilitation patients’ service needs. Our key findings were as follows: First, we discovered that the assessments made by physicians about rehabilitation patients’ needs did not align with those of experts. Second, we observed that the variation in assessments was influenced by the level of medical institution where the clinicians were employed.

The decision-making process for rehabilitation referrals mainly relies on the subjective evaluations of physicians, as there is currently no universally accepted gold standard for rehabilitation-tiered services. This results in assessment of rehabilitation service needs being somewhat subjective and inconsistent25. Furthermore, the reliability of clinicians’ assessments cannot always be accurately verified. Nonetheless, expert assessments are typically viewed as a more authoritative reference point26,27,28,29,30,31. It has been suggested that the gold standard for these assessments should be established by combining the consensus of at least three independent experts20,31. Consequently, this study used the assessment results of rehabilitation experts as a benchmark to explore the consistency between clinicians’ assessments and expert consensus regarding rehabilitation service needs.

In this study, 40 vignettes were randomly selected for evaluation. To minimize bias related to image clarity and patient information, the vignette images were standardized, and some basic patient details were omitted. The results indicate that the clinicians’ consistency with expert assessments in evaluating rehabilitation service needs is generally low. This finding suggests a significant gap exists between the diagnostic and treatment capabilities of clinicians compared those of experts. Previous studies have shown that experienced experts are more likely than general practitioners to correctly diagnose the same type of simple disease, and do so in less time32,33,34.

The reasons for this discrepancy are unclear; however, the baseline profile of physicians in this study may provide some insight. Lower levels of education and professional development among primary care physicians may affect their clinical decision-making and ability to conduct comprehensive rehabilitation assessments. Notably, a relatively high proportion of physicians in this study lacked specialty titles and had lower levels of education. This may indicate limited specialized training in rehabilitation medicine, potentially contributing to a lower consistency with expert evaluations28.

Additionally, we found that the consistency between clinicians’ and experts’ assessments increased with the level of the medical facility. This highlights potential limitations in the ability of primary care physicians to align their assessments with expert consensus. The particularly low concordance among clinicians in village health centres warrants further investigation. However, it should be noted that the number of clinicians from county hospitals was limited (n = 6), resulting in a wide confidence interval and greater statistical uncertainty in this subgroup. Therefore, the relatively high consistency rate observed at this level should be interpreted with caution.

Village doctors play an irreplaceable role in rural healthcare in China35. However, previous studies have pointed out that the diagnostic skills of rural physicians in China are often insufficient and suboptimal35,36. Despite the implementation of hierarchical diagnosis and treatment reforms in 2009, the performance of China’s primary healthcare system remains lacking in both process and outcomes37,38. Currently, primary care physicians in China are responsible for approximately 25% of all outpatient services and serve as gatekeepers within the broader healthcare system39. Given that, it is crucial to continue efforts to enhance the diagnostic and treatment abilities of primary care physicians. Strengthening their capacity to assess patients’ rehabilitation service needs should be a primary focus in future healthcare reforms.

Our findings in this study indicates that primary care doctors face challenges in accurately assessing rehabilitation needs, highlighting a critical gap in the current healthcare system. Strengthening rehabilitation capacity at the primary care levels is essential to address this issue9,40. As the first point of contact in the healthcare system, primary care institutions play a key role not only in identifying health conditions and initiating referrals, but also in delivering rehabilitation interventions and follow-up care39,41. Integrating rehabilitation into primary care supports a continuous, function-focused care model that improves quality of life, helps prevent disability, and promotes health equity. It also ensures better access to services in underserved areas and reduces pressure on high-level hospitals9,40. Therefore, strengthening rehabilitation training and resources at the primary level is essential to improve service delivery and meet patients’ needs more effectively42.

Strength and limitation

This study examined the consistency between clinicians and experts in assessing patients’ rehabilitation service needs, filling a gap in understanding the differences based on expertise levels. It also investigated how variations in healthcare institution types (primary vs. advanced hospitals) affect assessment outcomes, providing insights into resource distribution and service optimization. The findings offer valuable evidence for improving training, standardizing evaluation tools, and enhancing rehabilitation services across healthcare settings. However, there are several limitations to consider. First, the experts were recruited from our study team using a convenience sampling method, which may introduce selection bias and limit the generalizability of the findings. Expanding the expert pool through a more diverse and randomised selection process could improve the credibility and generalisability of the expert consensus used as the reference standard. Second, the sample of clinicians was unevenly distributed, with only six doctors from county hospitals, which could have introduced bias. This small sample size also led to a wide confidence interval for the consistency rate in this group, indicating high statistical uncertainty. As such, the results for the county hospital group should be interpreted with caution. Future studies should aim for a more balanced representation across healthcare institutions. Additionally, while expert consensus was used as the “gold standard”, it is inherently subjective and may reflect individual variability in clinical judgment. Although a structured consensus process was used to resolve disagreements, expert variability could still impact the results. Lastly, the 40 vignettes used in this study were limited in disease types, focusing predominantly on neurological and orthopedic conditions. A more diverse selection of vignettes, encompassing all major disease categories, would ensure a more comprehensive evaluation.

Conclusion

The ability of clinicians to assess patients’ rehabilitation needs is suboptimal, especially in primary healthcare settings (township health centers and village health centers). It is imperative to improve and standardise the ability of clinicians to assess the need for rehabilitation services.