Introduction

Medical services are valuable and limited resources. According to CDC statistics, the United States spent approximately $772.9 billion on outpatient services in 20191. However, about 12.0% of individuals under age 65 still could not access timely medical care due to the high cost2,3. Inefficiencies in outpatient clinics, reflected in long patient wait times, physician idle time, and overtime, stem from a mismatch between medical service demand and available capacity, further complicated by heterogeneous service times (i.e., consultation duration)4. Appointment scheduling (AS) involves deciding the sequence of patients for a physician’s service and allocating time slots for each patient5,6, which is widely used in outpatient clinics to match healthcare demand with providers’ capacity7. A well-designed and adaptable AS decision support system is vital to healthcare provider resource allocation and patient care quality8,9.

Since the pioneering work of Bailey10 in the 1950s, numerous vital aspects of outpatient AS have been investigated7,11,12. Most extant studies on outpatient AS model the patient service time as a homogeneous population, assuming either deterministic or independent and identically distributed (i.i.d.) consultation service time7. Such a simplified modeling choice allows researchers to study essential phenomena affecting the scheduling system, such as no-shows13,14,15,16,17,18, walk-ins19,20,21, and same-day re-entries22,23,24. However, such studies often overlook the heterogeneity of service times in real outpatient management practices. On the other hand, the performance of outpatient AS can be sensitive to patient heterogeneity, and therefore, it is beneficial to take patient heterogeneity information into account when developing AS systems25,26.

Categorizing patients based on their heterogeneity is essential for enhancing outpatient AS efficiency25,27. The heterogeneity in patient service times is addressed in various approaches in the literature, which can be divided into two distinct streams: (1) research modeling service times on an individual patient basis, acknowledging patient-specific heterogeneity28,29,30,31, and (2) studies considering groups of patients with different service time requirements24,25,32,33. The first stream of research considers the differences in service time at the individual patient’s level. However, implementing these strategies practically presents challenges. This is due to the need for high-quality historical data to accurately predict service times and no-show risk, along with a complicated AS system capable of adjusting allotted service time for individual patients. The second stream considers the differences in service time at the patient’s group level, in which the method of grouping patients is usually set a priori. For example, Cayirli et al.32 group patients based on whether they visit a facility for the first time for a condition. This is referred to as New/Return hereafter. The pre-defined classification simplifies scheduling implementation. Studies show that AS systems using simple pre-defined classification schemes can noticeably improve productivity34,35,36. However, pre-defined rules may not adequately capture the heterogeneity in patients’ service times.

One promising approach to addressing the above grouping issue is to utilize machine learning (ML) techniques, which bring benefits to improving outpatient operational efficiency37,38. ML excels at analyzing extensive data, uncovering relationships, and identifying patterns, making it valuable in medical and healthcare applications39,40,41,42. Additionally, ML allows computers to learn patterns autonomously without being explicitly programmed43, facilitating the exploration of a more promising outpatient AS scheme, particularly suited for contexts with heterogeneous service times. However, the classification scheme in the ML-related literature assumes that the grouping is given a priori or obtained through subjective experience, thus missing the opportunity to leverage ML to explore the value of historical data fully.

The limitations of existing research have driven us to develop an ML-based scheduling decision support system that balances implementation convenience and service time heterogeneity. Are there better ways to group patients regarding the number of groups used and how each group is defined? If yes, how can one find them, and how would one sequence patients based on these groups? To answer these questions, we propose a data-driven decision support system for outpatient AS in which the heterogeneity in patients’ service times is derived from empirical data. Such a system consists of three components, namely, Cluster, Predict, and Schedule (CPS). Initially, we cluster patients in historical data based on service times using unsupervised ML (i.e., K-median). Following the clustering process, we obtain each patient’s group label. We utilize service times from historical data alongside patient category labels and train a decision tree, a supervised ML model, to map each patient’s service time to their respective group. This decision tree will be employed as a classifier for incoming patient categorization. Next, we train a predictive model (i.e., XGBoost) to predict the incoming patients’ service time using existing patients’ data. The predicted service time is input into the decision tree to obtain the patient’s group label. Finally, upon obtaining category labels for all upcoming patients, scheduling ensues by implementing our proposed cost-minimizing sequence template based on patient groups.

We compare the proposed CPS to existing methods in the literature and practice and find that CPS is more cost-effective than competing methods. Compared to commonly used scheduling methods like FCFA (first-call, first-appointment) and New/Return-based approaches, our CPS method demonstrates cost reductions of approximately 15.0% and 4.7%, respectively. Moreover, our approach, CPS, demonstrates superior service fairness compared to FCFA. Specifically, our method results in smaller disparities in expected waiting times for patients scheduled across various time slots, in contrast to FCFA. The results and ensuing discussion offer valuable insights for healthcare institution administrators. They aid in improving AS decisions and applying ML techniques to reduce the operational costs of outpatient clinics.

Our study considers patient/service heterogeneity by utilizing data to balance too few groups (homogeneous assumption) and too much heterogeneity (individual-level differentiation). Classifying patients into manageable groups is realistic and easy to implement in medical practice25. The main contributions of our study are as follows:

  • To address the problems of low outpatient operational efficiency due to the heterogeneity of patient service times, this paper proposes a new data-driven method of patient classification based on service time clustering and prediction, using both unsupervised and supervised learning.

  • The general candidate sequencing rules, generated by our candidate rules generator (CRG) algorithm, yield significant efficiency improvement in numerical experiments using real-world data. Simple rules can perform close to the optimal appointment sequencing decision obtained by exhaustive enumeration in various clinical environments.

  • CPS is developed to determine the classification of patients using ML and to schedule patients based on the heterogeneity of their service times at the patient group level, thereby enhancing outpatient operational efficiency from these two perspectives.

The remainder of this paper is organized as follows. Section “Problem definition and methodology” describes our problem context, and outlines the proposed AS system, CPS. In Section “Results and discussion”, we delineate the implementation steps of CPS using real data from a clinical setting and derive its optimal schedule. Then, we present the numerical experiments, results, and derived insights. Finally, Section “Conclusions and future works” concludes and discusses our limitations and future research.

Problem definition and methodology

In this section, we describe the problem context and the proposed CPS system, including the mathematical formulation of the optimization problem and the workflow of the CPS.

The outpatient AS problem context

We consider the outpatient AS problem in the following general clinical setting. A clinic session of planned length \(\mathcal {T}\) (e.g., four hours) is divided into n equal-length slots. Each patient’s appointment is assigned to one slot. Scheduled patients arrive punctually at appointment times, and a single physician serves them with stochastic consultation times (i.e., the single-stage single-server setting). While a clinic may have multiple physicians, physicians usually have their own panels of patients. Therefore, one can schedule appointments for each physician independently. We assume the demand for service is large enough to fill all slots within each session consistently. This is supported by literature and by practice11,26,44. The n slots in a session are labeled 1 through n, indicating the chronological order. We label the appointments in a session according to their slot labels; thus, the \(i^{th}\) appointment is assigned to slot i. Hereafter, the \(i^{th}\) appointment and the \(i^{th}\) slot are used interchangeably.

To capture the cost of outpatient scheduling in our model, we adopt the notation (whenever applicable) from Chen and Robinson44 to define the following for slot i:

  • \(A_{i}\) is the appointment time. We set \(A_1=0\) by convention.

  • \(X_i \triangleq A_{i+1} - A_{i}\) is the service time allowance.

  • \(\xi _{i}\) is the random service time (whose distribution will be specified later).

  • \(Y_{i}\) is the realized start time of the \(i^{th}\) appointment with the physician. \(Y_1 = A_1 = 0\), and \(Y_{i+1} = \max (A_{i+1}, Y_{i}+\xi _{i}),\ i=1,...,n-1\).

  • \(W_{i}=Y_{i}-A_{i}\) is patient i’s waiting time.

We obtain the expression of the waiting using the famous Lindley recursion45:

$$\begin{aligned} W_{i+1}=(W_i+\xi _i-X_i)^+, \end{aligned}$$
(1)

where, \((x)^+ \triangleq max(x,0)\) is the non-negative part of any given x. \(W=\sum _{i=1}^{n} W_i\) is the total waiting time of all patients in the session.

I and O are the physician’s idle time and overtime, respectively. They can be determined as follows44:

$$\begin{aligned} \begin{aligned} I&= [Y_{n} - \sum _{i=1}^{n-1} \xi _i]^+ = [W_{n}+A_{n} - \sum _{i=1}^{n-1} \xi _i]^+, \\ O&= [\sum _{i=1}^{n} \xi _i+I -\mathcal {T}]^+, \end{aligned} \end{aligned}$$
(2)

Note that the appointment times \(\{A_{i}\}'s\) and thus \(\{X_i\}\)’s are fixed according to our assumption that the session is divided into n slots with fixed length. This is called individual-block fixed-interval AS in literature25,30,46.

Given a set of n patients to be scheduled for a session, we can assume that each one of them can be classified into one of the K classes for K being generic without loss of generality. This implies that if no classification is available, then \(K=1\); on the other hand, if each of the n patients has their own class, then \(K=n\). Let \(n_k\) be the number of patients of the \(k^{th}\) class for the session in question, \(k\in \{1,...,K\}\). Therefore, \(\sum _{k=1}^{K} n_k = n\).

An appointment schedule for a given session is the assignment of patients to each of the n available slots within that session. Our scheduling decision is to decide how to assign different patients to these n slots, and this is modeled by the decision variables \(\{x_{i,k}\}'s\). Each \(x_{i,k}\), for \(i\in \{1,...,n\}\) and \(k\in \{1,...,K\}\), is binary such that \(x_{i,k} = 1\) if the appointment assigned to slot i is a patient of class k; otherwise, \(x_{i,k} = 0\). Furthermore, as each patient from every class among these n patients needs to be scheduled, we have \(\sum _{i=1}^n x_{i,k} = n_k\) for each class k. Also, since each slot can accommodate only one patient; hence, \(\sum _{k=1}^K x_{i,k} = 1\).

Define \(\Xi _k\) as the random service time of class-k with the cumulative distribution function (CDF) \(\Phi _{k}(\cdot )\). The random service time of the appointment i, \(\xi _i \triangleq \sum _{k=1}^K \Xi _k\cdot x_{i,k}\) by definition, since the appointment i will be of a class-k for some \(k\in \{1,...,K\}\).

The optimization model

The objective function we used to evaluate any schedule is formulated as Eq. (3).

$$\begin{aligned} z = c_{W} E({\sum _{i=1}^n} W_{i})+c_{I} E(I)+c_{O} E(O), \end{aligned}$$
(3)

where, \(c_W\), \(c_I\), and \(c_O\) are the unit cost of patients’ waiting time (\(W_{i}\)), the physician idle time (I), and overtime (O), respectively. They can also be interpreted as the clinic’s perspective on the disutility of unproductive time. Furthermore, using \(W_{i}\), I, and O as key performance indicators (KPIs) for outpatient scheduling optimization is a common practice in the literature47,48,49.

With the problem setup stated in the previous section, the optimization model could be formulated as Model I.

$$\begin{aligned} \text{ min }&z = c_{W} E(\sum _{i=1}^{n} W_{i})+c_{I} E(I)+c_{O} E(O) \quad \quad \quad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad ({{\textbf {Model I}}}) \end{aligned}$$
$$\begin{aligned} \text{ s.t. } \nonumber \\&\xi _i = \sum _{k=1}^K \Xi _k\cdot x_{i,k},\; \forall i\in \{1,...,n\} \end{aligned}$$
(4)
$$\begin{aligned}&W_{i+1}=(W_i+\xi _i-X_i)^+, \; \forall i\in \{1,...,n-1\} \end{aligned}$$
(5)
$$\begin{aligned}&I = [W_{n}+A_{n} - \sum _{i=1}^{n-1} \xi _i]^+ \; \end{aligned}$$
(6)
$$\begin{aligned}&O = [\sum _{i=1}^{n} \xi _i+I -\mathcal {T}]^+ \; \end{aligned}$$
(7)
$$\begin{aligned}&\sum _{i=1}^n x_{i,k} = n_k,\; \forall k\in \{1,...,K\}\end{aligned}$$
(8)
$$\begin{aligned}&\sum _{k=1}^K x_{i,k} = 1,\; \forall i\in \{1,...,n\} \end{aligned}$$
(9)
$$\begin{aligned}&x_{i,k}\in \{0,1\},\; \forall i\in \{1,...,n\},\ \forall k\in \{1,...,K\}, \end{aligned}$$
(10)

Note that \(\{\Xi _k\}\)’s and hence \(\{\xi _i\}\)’s are random variables. Therefore, Model I is a stochastic programming problem. We further index the random variables using \(\omega\) where each \(\omega\) is a point from the probability space \(\tilde{\Omega }\). Model II is the ensuing formalization.

$$\begin{aligned} \text{ min }~&z = c_{W} E_{\omega }(\sum _{i=1}^n W_i(\omega )) +c_{I} E_{\omega }(I(\omega )) +c_{O} E_{\omega }(O(\omega )) \quad \quad \quad \qquad \qquad \qquad \qquad \qquad \qquad \qquad ({{\textbf {Model II}}}) \end{aligned}$$
$$\begin{aligned} \text{ s.t. } \nonumber \\&\xi _i(\omega ) = \sum _{k=1}^K \Xi _k(\omega )\cdot x_{i,k}, \forall i\in \{1,...,n\} \end{aligned}$$
(11)
$$\begin{aligned}&W_{i+1}(\omega )=(W_i(\omega )+\xi _i(\omega )-X_i)^+, \forall i\in \{1,...,n-1\}\end{aligned}$$
(12)
$$\begin{aligned}&I(\omega ) = [W_{n}(\omega )+A_{n} - \sum _{i=1}^{n-1} \xi _i(\omega )]^+ \; \end{aligned}$$
(13)
$$\begin{aligned}&O(\omega ) = [\sum _{i=1}^{n} \xi _i(\omega )+I(\omega ) -\mathcal {T}]^+ \; \end{aligned}$$
(14)
$$\begin{aligned}&\sum _{i=1}^n x_{i,k} = n_k, \forall k\in \{1,...,K\} \end{aligned}$$
(15)
$$\begin{aligned}&\sum _{k=1}^K x_{i,k} = 1, \forall i\in \{1,...,n\} \end{aligned}$$
(16)
$$\begin{aligned}&x_{i,k}\in \{0,1\}, \; \forall i\in \{1,...,n\},\ \forall k\in \{1,...,K\}, \end{aligned}$$
(17)

This is a stochastic programming problem with binary decision variables. Solving for the exact solution is computationally challenging. A common solution approach is Sample Average Approximation (SAA), i.e., randomly generating a sufficient number of samples and then optimizing the average cost of these samples19,50,51. So, we can take a large random sample of \(\omega 's\) from \(\tilde{\Omega }\) and replace the expectation operation above with the corresponding sample average. Let \(\Omega\) be the N sampled values of \(\omega\) (i.e., the number of sample instances \(|\Omega | = N\)), and the objective function becomes

$$\begin{aligned} \text{ min }~z = \frac{1}{N}\sum _{\omega \in \Omega } \{c_{W}\cdot \sum _{i=1}^n W_{i}(\omega ) + c_{I}\cdot I(\omega ) + c_{O}\cdot O(\omega )\}. \end{aligned}$$
(18)

Our appointment scheduling problem is then to optimize Eq. (18) with the constraints outlined in Eqs. (11) to (17), which is a mixed-integer programming (MIP) problem, and we call this Model III.

For given \(c_W\), \(c_I\), and \(c_O\), the optimal schedule is the sequence of different groups of patients (represented by \(\{x_{i,k}\}\)) that minimizes total cost z. If the whole patient population is treated as a homogeneous group, i.e., \(\{\xi _i\}_{i=1}^n\) are i.i.d., then there is no need to sequence them, which makes the computational problem trivial but compromises scheduling productivity. The other extreme is considering heterogeneity at the individual level, i.e., \(K=n\). This may fully utilize the heterogeneity in service time and maximize the scheduling benefit, but the computational effort needed to sequence all n patient classes will become forbidding since there will be n! different sequences. Our approach is to strike a balance between these two extremes. We classify patients into a few groups and model the service times of patients within each group as i.i.d. random variables. The proposed decision support system will make our scheduling problem computationally more tractable and practically more manageable. The proposed framework for classifying and scheduling patients will be discussed in the next section.

The CPS framework

CPS’s workflow is illustrated in Fig. 1. In Stage 1, leveraging historical data, we utilize unsupervised ML to cluster patients based on their service times. To explicitly capture the clustering mechanism, we train a decision tree to learn the mapping from a patient’s service time to the corresponding cluster. We could derive the cutoff point(s) from the resulting decision tree. In Stage 2, we train the supervised ML model that predicts a patient’s service time based on the patient’s characteristics. The predicted service time is input into the decision tree in the first stage, and the patient’s group is the output. Then, in Stage 3, we first propose a scheme to generate candidate schedule templates. A schedule template is the assignment of patients of different groups to the slots of a clinic session. We then use empirical data to conduct Monte Carlo simulations to pick the best templates (i.e., the output of the CPS) for various clinic settings. In the rest of this section, we detail the processes in CPS.

Fig. 1
Fig. 1The alternative text for this image may have been generated using AI.
Full size image

A flowchart of the CPS system. A rectangular represents a process. A hexagon represents some form of data. Moreover, a hexagon with a filled background signifies the input or output within this schematic representation. Stage 1: Cluster. Leveraging historical data, we utilize unsupervised ML to cluster patients based on their service times. To explicitly capture the clustering mechanism, we train a decision tree to learn the mapping from a patient’s service time to the corresponding cluster. Stage 2: Predict. We need the service time of each incoming patient to use the cluster. So, we train the supervised ML model that predicts a patient’s service time based on the patient’s characteristics. Stage 3: Schedule. We first propose a scheme to generate candidate schedule templates. A schedule template is the assignment of patients of different groups to the slots of a clinic session. We then use empirical data to conduct Monte Carlo simulations to pick the best templates (i.e., the output of the CPS) for various clinic settings.

Cluster patients based on historical service times

The initial stage of CPS involves employing unsupervised ML techniques to cluster patients, capturing the heterogeneity in their service times. We cluster based on patient service times (i.e., \(\xi _i\)’s) because they directly influence AS’s effectiveness and the objective value z, as indicated in Eqs. 1 to 3, rather than patient characteristics. To obtain patients’ cluster labels, we apply K-median clustering, less outlier-sensitive than K-means, to historical patients’ service times, indicated by the purple components in Fig. 1. We then utilize a decision tree to learn the clustering rules, i.e., the unsupervised learning results, indicated by the orange components in Fig. 1.

Predict incoming patients’ service times

Clustering in CPS requires the knowledge of the incoming patient’s service time, which is the future information we would not have when scheduling the appointment. As such, we use the predicted service time as a proxy. Specifically, we utilize historical data on patients’ characteristics and the corresponding service times to train a supervised ML model for predicting incoming patients’ service times. The patients to be scheduled (i.e., incoming patients) are then classified using their predicted service times as the input to the fine-tuned decision tree obtained in the previous section. These processes are illustrated by the dark blue and light blue components in Fig. 1, respectively.

Schedule incoming patients groups

Then, we design the sequencing rules to schedule the acquired patient groups. These processes are illustrated by the red components in Fig. 1. Generally, the design of sequencing rules is considered one of the most important aspects of AS. There are well-known sequencing rules when K, the number of patient groups, is 2 (e.g., Type A and Type B) in the literature25,30; and we call these rules the traditional sequencing candidate (TSC) rules. The following five scheduling rules are frequently employed in such cases.

  • ABG: assigning Type A patients to the beginning slots and Type B the remaining slots25,30,32,52,53,54,55.

  • BBG: the reverse of ABG25,30,32,52,53,54,55.

  • ABND: assigning Type B to the middle slots and leaving both the beginning and the ending slots to Type A25,30.

  • BBND: assigning Type A to the middle slots and leaving both the beginning and the ending slots to Type B25,30.

  • ALTER: alternating between Type A and Type B25,32,52,53,54.

These TSC rules are easy to implement in practice but cannot be directly generalized to handle cases beyond two patient types. To the best of our knowledge, for \(K \ge 3\), the literature provides no generic sequencing schemes beyond sorting the appointments by the expectation or the variance of patient groups’ service times26.

To address this, we develop a generic algorithmic scheme, named CRG, to generate candidate schedule sequences for any K. We use \(\texttt {TYPES} =(T_1, T_2,\ldots , T_K)\) to denote the K types of patients, and \(\textbf{c}=(n_1,n_2,\ldots ,n_K)\) is called a composition, indicating that there are \(n_k\) patients of type \(T_k,\ k=1,2,\ldots ,K\), to be scheduled. Given \(\texttt {TYPES}\) and \(\textbf{c}\), the candidate templates set \(\texttt {Templates}\) can be derived using the CRG algorithm. We can further evaluate the candidate templates and pick the best one for a clinic. More details about CRG are provided in the following section.

The CRG algorithm

The main idea of CRG is to concatenate two types of sub-sequences: the one with repeating patterns and the one with remaining patients not included in the repeating patterns. The precise meaning of repeating patterns and the definition of other necessary terms are given below.

As mentioned above, let \(\texttt {TYPES} =(T_1, T_2,\ldots , T_K)\) denotes the K types of patients. Let \(\mathbb {Z}_+\) and \(\mathbb {N}\) be the set of non-negative and positive integers, respectively. \(\textbf{c}=(i_1,i_2,\ldots ,i_K)\in \mathbb {Z}^K_+\) is called a composition of patients, which represents \(i_k\) patients of type \(T_k\) for \(k=1,2,\ldots ,K\). A composition \(\textbf{c}\) such that \(\textbf{c}\in \mathbb {N}^K\) is called a positive composition. We do not further differentiate patients beyond patient types, and thus \(\texttt {TYPES}\) together with a composition \(\textbf{c}\), i.e., \((\texttt {TYPES},\textbf{c})\), defines a set of patients. A sequence of \((\texttt {TYPES},\textbf{c})\) is to put all the patients of the set into some order.

Suppose we originally have the patient set \((\texttt {TYPES}, \textbf{c})\), and \(\mathbf {c'}\) is another composition such that \(\mathbf {c'}\le \textbf{c}\) in the vector sense, then \(\textbf{c}-\mathbf {c'}\) also denotes a composition, which represents the patients in \((\texttt {TYPES}, \textbf{c})\) but not in \((\texttt {TYPES}, \mathbf {c'})\). Note \(\textbf{c}-\textbf{c}\equiv \textbf{0}\) represents an empty set of patients by definition. We say a sequence is Type-Bundled if patients of the same type are arranged next to each other in this sequence. Given the patient set \((\texttt {TYPES},\textbf{c})\), we use \(\texttt {TypeBundles}(\texttt {TYPES},\textbf{c})\) to denote the collection of all Type-Bundled sequences of this patient set.

By definition, if \(\textbf{c}\) is a composition, so is \(r\cdot \textbf{c}\), \(\forall r\in \mathbb {N}\). If \(\textbf{p}\) is a sequence of patients in the patient set defined by \((\texttt {TYPES},\textbf{c})\), we use \(r\otimes \textbf{p}\) to denote the sequence that repeats \(\textbf{p}\) for r times; therefore \(r\otimes \textbf{p}\) is a sequence of patients in the set \((\texttt {TYPES},r\cdot \textbf{c})\). For the trivial case of \(r=1\), \(1\otimes \textbf{p}\equiv \textbf{p}\). For a sequence \(r\otimes \textbf{p}\), we call \(\textbf{p}\) a repeating pattern.

Algorithm 1
Algorithm 1The alternative text for this image may have been generated using AI.
Full size image

Candidate Rules Generator (CRG).

With the terms defined above, Algorithm 1 describes our scheme that generates the candidate appointment sequences for the session of \(\sum n_k\) slots with the set of patients \((\texttt {TYPES},\textbf{c})\) such that \(\textbf{c}=(n_1,\ldots ,n_K)\). Here is the explanation of the algorithm. For each positive composition \(\mathbf {c'} \le \textbf{c}\):

  • Step 1. Generate Type-Bundled sequences to form \(\texttt {Patterns}\), the set of pattern sub-sequences (line 4).

  • Step 2. Determine \(r = \min _{k} \lfloor \frac{n_k}{i_k}\rfloor\) (line 5), the maximum times a pattern can be repeated.

  • Step 3. Construct \(\texttt {RepeatingPatterns}\), the collection of sub-sequences that repeat the sub-sequences of \(\texttt {Patterns}\) for r times (line 6).

  • Step 4. For the remaining patients represented by the composition \(\textbf{c}-r\cdot \mathbf {c'}\), construct the corresponding collection of sequences \(\texttt {Remainders} {\mathop {=}\limits ^{\text {def}}}\texttt {TypeBundles}(\texttt {TYPES}, \textbf{c}-r\cdot \mathbf {c'})\) (line 7).

  • Step 5. Obtain a new collection of schedule sequences for the session, \(\texttt {NewRules}\). Each sequence in it is the concatenation of one sub-sequence from the repeating pattern collection \(\texttt {RepeatingPatterns}\) and a sub-sequence from the remaining patients’ collection \(\texttt {Remainders}\). We use \(\oplus\) to denote the concatenation of two sub-sequences (line 8).

  • Step 6. Update the candidate rules set \(\texttt {Templates}\) by adding \(\texttt {NewRules}\) (line 9).

We also provide an illustrative example to demonstrate the functionality of CRG in Supplementary Appendix A.

Templates evaluation

Lastly, the candidate templates can be evaluated using the associated cost z, as defined in Eq. (18), considering different clinic preferences represented by \(c_W\), \(c_I\), and \(c_O\). In literature, a frequently used technique is approximating expectation by sample average, also known as Monte Carlo simulation. We adopt the idea of SAA but sample the service times \(\{\xi _i\}'s\) from the historical data to evaluate the candidate templates and pick the best one. Once the sample service times are given, we can use the Eqs. (11) to (17) to compute \(W_i\), I and O for a given template (with its sequencing decision represented by \(\{x_{i,k}\}'s\)).

Results and discussion

This section presents the numerical results and discusses the important findings.

Case study

To showcase how CPS works, we conducted numerical experiments using a real dataset from our partner clinic and its operational context, with \(n=16\) (i.e., a session comprising 16 slots over four hours). The same approach and principles can be applied in other clinics using their specific data sets.

The real-world problem

Hangu is our partnered clinic in Guangzhou, China. At Hangu, patients make appointments either by phone or online, and they receive one appointment slot for each consultation. The clinic currently adopts a FCFA approach for scheduling. FCFA, while straightforward, is widely implemented across various clinics and hospitals26,32,56. Nonetheless, FCFA overlooks the heterogeneity in patient service times, potentially leading to operational inefficiencies.

The publicly available dataset57 used in our study contains 6637 consultation records collected from 381 half-day sessions of its stellar physician between 2018 and 2019. The dataset is evenly divided based on the entry date: the training set comprises records from January 2018 to January 2019 (50% of the dataset, totaling 3319 records), which is appropriate to train the model and meets the requirements for achieving satisfactory performance58,59. The test set includes records from January to December 2019 (50% of the dataset, totaling 3318 records).

We performed the feature pre-processing before using the data to construct the CPS model used for Hangu. All continuous variables undergo standardization, while categorical variables undergo one-hot encoding. The feature preprocessing steps occur after splitting the dataset into training and testing sets. This ensures that scaling parameters only come from the training set, preventing information leakage from the test set. While the details of the data can be found in the data article, Feng et al.56, Table 1 presents the feature variables’ meaning and statistical descriptions of the data. Besides the features, there is a response variable for the supervised ML model to predict, which is ServTime, the service time recorded. Note that there are 2392 missing values in the Address feature, attributed to patient preferences, which carries information related to patient characteristics. Additionally, the regression analysis indicates that Address_NA coefficients strongly predict ServTime (\(p<0.001\), two-tailed t-test) via linear regression56. Thus, we retain these missing values as a distinct category within the feature, refraining from imputation or exclusion.

Table 1 Variable meanings and their statistical descriptions. †We assume that the clinic scheduler has access to patient-related features prior to their appointment. This assumption is supported by literature and medical practices26,60. These practices often require patients to complete account registration and provide personal details before requesting an appointment. ‡Missing values occur only in the His.ServTime and Address features, with missing rates of 6.07% and 36.04%, respectively. For the His.ServTime feature, we impute the missing values with the length of one slot (i.e., 15 minutes). For the Address feature, we treat the missing values as a distinct category.*Mean(SD) for continuous features and N(%) for categorical features.
Full size table

The data will be used as the input of the CPS model. We first use the training set to train the clustering and service time prediction models. These models are then applied to the test set to determine each patient’s type in the test set. We then generate schedule templates and evaluate them using the test set. The details are given in the following subsections.

Clustering patients

We employ the Python package pyclustering61 on the training set with the default parameters for the clustering. The quality of the clustering is measured by the sum of absolute error (SAE) and the silhouette coefficient (SC)62, as shown in Figures 2a and 2b respectively. Both \(K=2\) and \(K=3\) exhibit an inflection point in their SAE values, as shown in Fig. 2c (elbow plot). Moreover, \(K=3\)’s SAE is lower, indicating better clustering in terms of SAE. \(K=2\)’s SC is higher, indicating better clustering in terms of SC.

Fig. 2
Fig. 2The alternative text for this image may have been generated using AI.
Full size image

Clustering quality metrics: (a) sum of absolute errors (SAEs) and (b) silhouette coefficients (SCs). (a) and (b) are the plots of the SAEs and the SCs, respectively, against the number of clusters K. The SC quantifies the compactness and separation of clusters based on the distance between data points within and between clusters. It ranges from -1 to 1. Higher values indicate better clustering.

We further compute the 95% confidence intervals of SAE and SC for a closer examination; see Table 2. The result shows that both differences are statistically significant. Since both \(K=2\) and \(K=3\) have merits, we include both in our study. The decision trees we built to capture the clustering rules are shown in Fig. 3.

Table 2 Clustering performance in terms of internal validation metrics. The performance values and the 95% confidence intervals (in parentheses) for the clustering were obtained from 1000 bootstrap resamples of the training set.
Full size table
Fig. 3
Fig. 3The alternative text for this image may have been generated using AI.
Full size image

Decision trees: (a) \(K=2\) and (b) \(K=3\). The service classification decision tree is based on K-median clustering. When \(K=2\), without loss of generality, we label the cluster with the larger mean service time A. Figure 3a shows the resulting tree structure. The cut-off point of the service times of patients in the two clusters is 811.5 seconds. When \(K=3\), we label the clusters A, B, and C in descending order of the mean service time of these patients. The decision tree for this clustering rule is shown in Fig. 3b, whose cut-off points are 630.5 and 975.5.

Predicting service times

In this section, we build predictive models using the training set for predicting incoming patients’ service time in the test set to put them into groups we obtained in the previous step. We employ linear regression (LR), support vector regression (SVR), extreme gradient boosting (XGBoost), and deep neural network regression (DNNR) to predict patients’ service time. These methods are widely adopted and robust in predicting uncertainties associated with outpatient scheduling30. Our implementation relies on the scikitlearn framework (for LR, SVR, XGBoost) and keras (for neural networks). All ML components are implemented with Python. Table 1 presents the input features utilized by these models. We employ 10-fold cross-validation for training and validation purposes. Bayesian optimization is used for tuning the hyperparameters. A comprehensive list of all hyperparameters explored and their respective search ranges can be found in Table 3.

Table 3 Hyperparameters of each predicting model.
Full size table

We use root mean squared error (RMSE), and \(R^2\) to evaluate the performance of each model, and results in Table 4 show that XGBoost generally outperforms others. Consequently, XGBoost is selected as the predictive model, with its feature importance values (%) ranked in descending order: His.ServTime (32.7), Visit.No (25.1), M.Cancer (15.2), Address (13.5), Return (9.2), S.Cancer (2.7), and Gender (1.6).

Table 4 Performance of each predicting ML model. The performance metrics of the model and the 95% confidence intervals (in parentheses) were obtained by performing 1000 bootstrap resamples on the test set.
Full size table

Patient classification and schedule evaluation

After we predict the incoming patients’ service times, we use the decision trees in Fig. 3 to obtain patients’ cluster labels. Next, we generate candidate schedule sequences using CRG under a specific classification scheme. We then select the best sequence for each chosen setting through simulations conducted with the test set. Besides the proposed classification scheme, two popular alternatives from the literature are also adopted for comparison to gain management insights. These two alternatives are FCFA and New/Return. Notably, FCFA represents the current practice at Hangu and serves as our baseline.

To generate schedule sequences for comparison, we need to determine the set of patients \((\texttt {TYPES},\textbf{c})\) for each session. For each classification scheme (excluding New/Return), \(\texttt {TYPES}\) is given by the decision tree; the patient composition \(\textbf{c}\) is determined in a way that maintains fair access to care for patients regardless of the classifications. This means the long-run percentage of slots for each patient group is proportional to the percentage this group takes up in our patient population. The detailed patients classification is presented in Table 5. We document how to obtain these numbers in each scenario in Supplementary Appendix B.

Table 5 Patients classification when \(K=2\) and \(K=3\). In each session of a specific physician, a fixed number of 16 patients are scheduled. In the case of New/Return, the New and Return are labeled as Type A and Type B, respectively.
Full size table

Given the patient set \((\texttt {TYPES},\textbf{c})\) (of 16 patients), we can use CRG to generate a set of candidate sequences, namely \(\texttt {Templates}\), for their appointment. For each sequence in the \(\texttt {Templates}\), we use the average of 10, 000 replications (sampling from the test set) to evaluate it, as described in Section “Problem definition and methodology”. Due to space limitations, additional details are provided in Supplementary Appendix C. In our simulation, \(c_W\) is normalized to 1 without loss of generality, \(c_I\in \{0, 5,10\}\) and \(c_O\in \{1,2,...,10\}\), resulting in \(3 \times 10=30\) instances. These parameters cover typical clinic settings in the literature and extend beyond specific clinics like Hangu30,63,64. Then, we select the sequence with the lowest cost z as the best sequence in each clinic preference setting. Therefore, each policy’s solution schedule is the collection of sequences, each being the best sequence found by simulation under one scenario. The corresponding objective z is the weighted average of the \(z's\) obtained under different scenarios listed in Table 5. A detailed explanation is available in Supplementary Appendix D.

Experimental setup

This section outlines our experimental setup, comparing CPS’s performance with Hangu’s FCFA, and New/Return+TSC, moment-related (MR) rules, which are from the literature.

As discussed in the previous section, we incorporate both \(K=2\) (referred to as K2) and \(K=3\) (referred to as K3) in our study due to their respective strengths. When patients are classified into two pre-defined categories, such as New/Return, there are well-known TSC rules. When using our data-driven classification scheme with K2, we use TSC as an alternative sequencing method and compare its performance with the one of the proposed CRG. When K3, TSC is unavailable as it only applies to two-type classifications.

In addition to TSC rules, three other promising sequencing algorithms for AS are mentioned in the literature50,65. These rules, defined by the first and/or the second moment of service duration, are collectively referred to as MR.

  • Smallest-mean-first (SMF). Sequence patients in order of increasing mean of service durations50,55,66.

  • Smallest-variance-first (SVF). Sequence patients in order of increasing variance of service durations67,68,69.

  • Smallest-coefficient-of-variations-first (SCVF). Sequence patients in order of increasing coefficient of variation (CV) of service durations. The CV is defined as the ratio of the standard deviation to the mean, namely \(CV=\sigma /\mu\)64,65,70.

Previous studies show no single sequencing method is superior across all potential settings. While some research suggests that sequencing by increasing processing times yields the best results65,66, other studies propose that scheduling longer appointments first can be more effective5,32. Furthermore,71 find that, in certain cases, prioritizing patients with higher service duration variability can surpass the SVF rule’s performance. Therefore, we conducted experiments that included SMF, SVF, SCVF, and their respective reverse forms. The MR rules for these comparative experiments were derived from the training set data of our study, as detailed in Table 6.

In the meantime, one may also exhaustively enumerate all the sequences for a specific set of patients with a given classification (Enum) for K2 and K3, respectively. This is essential to solve Model III to the optimum by enumerating all the possible \({x_{i,k}}'s\) and pick the one with the best z in Eq. (18) for any given combination of \(c_W\), \(c_I\), and \(c_O\). While this approach could be time-consuming and result in excessive candidate sequences, we still choose to include Enum for K2 and K3 in our experiment for benchmarking purposes. Note that we can also solve Model III to the optimum by popular commercial solvers, such as Gurobi, and interested readers are referred to Supplementary Appendix E for the detailed discussion about the computational efforts as well as solution quality.

Table 6 Moment-related (MR) sequencing rules. B\(\rightarrow\)A implies that patients of Type B are scheduled before patients of Type A. The rules in parentheses represent the reversed forms.
Full size table

As a result, to investigate the effects of classification and sequencing, we evaluate the following policies (combinations of classification and sequencing schemes) besides the baseline FCFA: (1) New/Return+TSC; (2) K2+TSC; (3) K2/K3+CRG; (4) K2/K3+Enum; (5) New/Return+MR; and (6) K2/K3+MR.

The proposed CRG algorithm, TSC, and MR rules were implemented using R 4.2.3 on the Windows 11 platform. The experiments were conducted on a laptop with an AMD Ryzen 7 6800H CPU running at 3.20 GHz and equipped with 16 GB of RAM. In the computational experiments, all the Model III instances were solved by calling Gurobi 11.0.0 on R 4.2.3. When using Gurobi, we employed its default termination parameters (i.e., TimeLimit = Infinity, MIPGap = 1e-4, MIPGapAbs = 1e-10, NodeLimit = Infinity).

Performance evaluation

For each scheme of interest, we report the relative performance measure with respect to FCFA’s, \(\frac{z}{z_{F}}\), where z represents the simulated performance measure of each scheduling scheme (excluding FCFA) of Eq. (3), and \(z_{F}\) is the FCFA counterpart serving as the baseline. \(\frac{z}{z_{F}} < 1\) indicates that the current scheme outperforms FCFA, and methods with lower values of \(\frac{z}{z_{F}}\) are favorable. Figures 4 and 5 show the comparisons in schemes considered with various combinations of \((c_I,c_O)\) while \(c_W\) is normalized to 1.

Overall, all methods mentioned in this paper, except MR, outperform FCFA (i.e., their \(\frac{z}{z_{F}}<1\)), regardless of the clinic’s preferences concerning the disutility of physician idling and overtime. MR also outperforms FCFA most of the time, with a few exceptions when physician idling cost is set to zero and physician overtime cost is at a moderate level. In particular, CPS exhibits a significant cost reduction compared to FCFA, the current practice at Hangu as well as many other clinics. Savings of nearly \(15.0\%\) can be achieved using K2+CRG at \((c_I,c_O)=(0,1)\) compared to FCFA, as shown in Fig. 4a; similarly, savings of nearly \(14.0\%\) are observed with K3+CRG at \((c_I,c_O)=(10,10)\), as shown in Fig. 4b. In addition, both K2+CRG and K3+CRG’s solutions are close to the optimal solution provided by K2+Enum and K3+Enum, respectively. In the following, we compare CPS to New/Return+TSC, MR, and a commercial solver, Gurobi’s solution.

Fig. 4
Fig. 4The alternative text for this image may have been generated using AI.
Full size image

Cost comparison of CPS and New/Return+TSC methods: (a) Comparing New/Return+TSC with K2, (b) Comparing New/Return+TSC with K3 and (c) Comparing K2+CRG and K3+CRG.

CPS vs. New/Return+TSC

When compared against the widely-used New/Return+TSC in literature, the savings of CPS can be more than \(4.7\%\) (when using K3+CRG at \((c_I,c_O)=(5,10)\)) , as shown in Fig. 4b. If we fix the number of patient types at two, K2+CRG can save 4.12% compared to New/Return+TSC, as shown in Fig. 4a at \((c_I,c_O)=(0,2)\). We provide a detailed discussion in the following.

Figure 4a compares the best solution found by New/Return +TSC and the ones found by using our classification for K2 combined with different sequencing schemes. K2+TSC outperforms New/Return+TSC in most \(c_I\) and \(c_O\) configurations; the savings can reach 3.06% at \((c_I,c_O)=(0,6)\). The implication is that a better classification scheme using data (i.e., K2) yields a better scheduling performance than the pre-defined New/Return. In addition, K2+CRG outperforms K2+TSC, indicating that searching over the better sequencing candidates obtained by CRG yields further improvement; the savings can reach 5.73% at \((c_I,c_O)=(0,1)\). The costs of K2+CRG’s solutions are the closest to the optimal ones obtained by K2+Enum. The cost gap can range from 0.22% to 1.20%.

Figure 4b compares our method of \(K\textit{3}\)+CRG with New/Return+TSC. As mentioned in Section “Problem definition and methodology”, when the number of patient categories is three or more, the scheduling method of TSC is no longer applicable, whereas CRG remains capable of addressing this issue. Therefore, we compare K3+CRG to New/Return+TSC and observe a notable reduction in costs, with savings reaching \(4.71\%\) at \((c_I,c_O)=(5,10)\). The costs of K3+CRG’s solutions are the closest to the optimal ones obtained by K3+Enum. The cost gap can range from 0 to 0.68%.

Another observation drawn from Figs. 4a and 4b is that, for both K2 and K3, one only needs to search over the candidate templates generated by CRG to achieve a performance close to that of enumerating all conceivable sequence templates (Enum). This is particularly true in K3, as K3+CRG performs nearly as well as K3+Enum. This highlights the advantage of CRG for practitioners since a full enumeration of all possible templates is computationally forbidding and practically cumbersome.

Figure 4c compares a simpler classification (K2) scheme with the one using an additional category (K3). The results show that an increased number of categories in patient classification generally reduces overall costs by up to 2.61%. However, such nominal cost savings come at the expense of increased management complexity.

CPS vs. Moment-related methods

Figure 5 shows that CPS is also superior to MR. Figure 5a indicates that CRG outperforms or equals MR when using advanced patient classification methods (K2 and K3). Specifically, when K2 is used, CRG significantly outperforms MR in all instances; when K3 is used, CRG significantly outperforms MR when \(c_I=0\), while performance is identical with MR in the remaining \(c_I\) cases. Figure 5b suggests that in cases where patient classification relies on New/Return, CRG demonstrates enhanced performance in patient sequencing compared to MR across all instances. Figure 5c indicates that, across all instances tested, CPS outperforms New/Return+MR, which is the combination of popular patient classification scheme New/Return together with MR for appointment sequencing. Specifically, K2+CRG outperforms New/Return+MR in all \(c_I\) and \(c_O\) configurations, with savings reaching 8.37% at \((c_I,c_O)=(0,5)\). Similarly, K3+CRG dominates New/Return+MR in all \(c_I\) and \(c_O\) configurations, with maximum savings up to 9.44% achieved at \((c_I,c_O)=(0,5)\). This suggests that CPS has a significant advantage over state-of-the-art methods in reducing outpatient scheduling costs.

Fig. 5
Fig. 5The alternative text for this image may have been generated using AI.
Full size image

Cost comparison of CPS and MR: (a) Comparing CPS with K2+MR and K3+MR, (b) Comparing New/Return+CRG with New/Return+MR and (c) Comparing CPS with New/Return+MR.

Fairness analysis

Besides the costs, the fairness of patient service interests practitioners and researchers72. It is well-known in literature and practice that patients assigned to later appointment times of the session generally experience longer waiting times due to the congestion that tends to build up over time7. One metric to measure this disparity is the standard deviation of the average waiting time throughout a clinic session32,73, and we denote it by STD(W). Scheduling policies of small STD(W) are considered more equitable. Figure 6 plots the relative performance of each policy in terms of STD(W) using FCFA’s as the benchmark (\(\textit{STD}_F(W)\)). It shows that the schedules obtained by CPS aiming to optimize the overall costs are also desirable in terms of STD(W). In particular, the solution found by K3+CRG performs consistently well in terms of STD(W) across all clinic cost preferences. Furthermore, as shown in Fig. 6a, K3+CRG exhibits greater stability in fairness compared to K2+CRG and New/Return+TSC, consistently maintaining a higher level of fairness (low STD(W)). Regardless of the patient classification method used, the fairness of MR falls short compared to CRG, with K3+CRG exhibiting the highest level of fairness among these strategies, as illustrated in Fig. 6b.

Fig. 6
Fig. 6The alternative text for this image may have been generated using AI.
Full size image

The disparity in patient waiting times: (a) The disparity in patient waiting times of CPS and TSC and (b) The disparity in patient waiting times of CPS and MR.

Additionally, we evaluate fairness among various genders and address groups, which is an essential demographic characteristic in our study (as shown in Table 1). The experimental results show that our CPS system does not show favoritism towards any particular gender or address group, thereby meeting the fairness requirements in healthcare services. Detailed results can be found in Supplementary Appendix F.

Managerial implications

This research provides valuable managerial insights tailored for outpatient management practitioners. It aims to support them in reducing operational costs while improving the patient experience.

First, outpatient practice managers can leverage CPS to trim operational patient classification and scheduling costs. These cumulative benefits allow a phased implementation of CPS to mitigate complexity. For instance, if adjusting the scheduling system poses challenges, adopting CP (Cluster and Predict) components of CPS independently from S (Schedule) can enhance patient classification, followed by using the existing TSC scheduling method. Furthermore, the simulation results can guide a partial CPS implementation based on cost settings, curbing unnecessary management expenses. For example, excluding physician idle time costs and with moderate overtime expenses (5 to 6 times the patient wait cost), implementing K2+TSC can yield results similar to K2+CRG without altering the scheduling scheme.

Second, CPS demonstrates favorable performance regarding equity of patient visits, enhancing patient satisfaction. Since CPS does not differentiate among patients of the same category, it can, to a certain extent, accommodate patient preferences for different time slots in outpatient practice. Simulation results indicate that, in most instances, the fairness of CPS is superior to the traditional New/Return+TSC method. Notably, as classifications become more detailed, the fairness performance of CPS consistently outperforms FCFA across various outpatient settings.

Lastly, increasing the number of patient categories may not always lead to reduced operational costs. Specifically, when the cost of physician idle time is disregarded, and overtime expenses remain low (less than four times the patient unit wait time cost), in particular settings that prioritize patient experience, such as boutique clinics or concierge medicine, the utilization of the K2+CRG approach appears more advantageous than K3+CRG in reducing outpatient operational costs. However, within healthcare institutions that prioritize reducing physician idle time and overtime expenses (e.g., specialist clinics), a more detailed classification can reduce outpatient operational costs but add complexity to outpatient management. Thus, achieving an optimal balance requires careful consideration of the context within outpatient management.

Conclusions and future works

This study provides a novel data-driven decision support system for redesigning scheduling templates at the patient group level. Patients are first grouped and then scheduled in slots based on their group labels. The simulation results show a significant reduction in the clinic’s cost, considering the patient waiting times, physician idle time, and overtime, while improving patient equity in terms of waiting time disparity. The proposed templates are also easy to implement and interpretable. They could assist hospital or clinic administrators in refining and optimizing AS decision-making processes through adaptive strategies that maximize the utilization of medical resources. Moreover, CPS could periodically update its models to learn from updated historical data, enabling dynamic adaptation to changing circumstances.

Compared to existing scheduling methods, CPS has clear advantages. It uses data-driven patient classification to improve scheduling over traditional methods like New/Return while reducing operational costs and ensuring equitable access to healthcare resources. In terms of sequencing, the benefits from patient classification allow CRG to efficiently generate optimal or close-to-optimal schedule templates compared to commercial solvers, delivering comparable performance. However, CPS has limitations. It requires ML for patient classification, making the classification less intuitive than the traditional ones (e.g., New/Return). The templates produced by CRG, though containing repetitive patterns, may be more sophisticated than those intuitive rules (e.g., FCFA, TSC), demanding refined management capabilities for outpatient scheduling.

To the best of our knowledge, this research represents the first attempt to combine supervised and unsupervised ML techniques to create patient classifications and develop a novel CRG algorithm for more efficient patient scheduling with heterogeneous service times. While this study focuses on outpatient clinics, the proposed AS decision support system, namely CPS, also applies to other medical services with appointment-based arrivals and heterogeneous duration, such as day surgeries and psychological counseling. Our study excludes patient no-shows due to unavailable records in the showcase clinic. Future research can enhance CPS by considering more complex settings, such as no-shows and re-entrance. While our case study utilizes a real dataset from a collaborative clinic, our CPS system is designed to be adaptive and thus not confined to any specific dataset or clinic environment. Any clinic can readily implement it with access to historical data on service durations and associated patient information such as demographics, reasons for visits, and past appointment history. Additionally, our numerical studies accommodate a broad spectrum of clinic preferences concerning patient and physician productivity, as demonstrated by the varied values of \(c_W\), \(c_I\), and \(c_O\). Future research could further enhance the productivity improvements of the CPS by incorporating datasets that include more comprehensive patient information, should such datasets become available.