Optimal dosing of anti-cancer treatment under drug-induced plasticity

Abstract

While cancer has traditionally been considered a genetic disease, mounting evidence indicates an important role for non-genetic (epigenetic) mechanisms. Common anti-cancer drugs have recently been observed to induce the adoption of non-genetic drug-tolerant cell states, thereby accelerating the evolution of drug resistance. This confounds conventional high-dose treatment strategies aimed at maximal tumor reduction, since high doses can simultaneously promote non-genetic resistance. In this work, we study optimal dosing of anti-cancer treatment under drug-induced cell plasticity. We show that the optimal dosing strategy steers the tumor to a fixed equilibrium composition between sensitive and tolerant cells, while precisely balancing the trade-off between cell kill and tolerance induction. The optimal equilibrium strategy ranges from applying a low dose continuously to applying the maximum dose intermittently, depending on the dynamics of tolerance induction. We finally discuss how our approach can be integrated with in vitro data to derive patient-specific treatment insights.

Introduction

Cancer is the result of a complex evolutionary process which results in cells gaining the ability to divide uncontrollably^1,2. While genetic mutations have long been understood to be important drivers of cancer evolution, it has more recently become clear that epigenetic mechanisms such as methylation and acetylation are sufficient to fulfill the traditional hallmarks of cancer^3,4,5,6. Epigenetic mechanisms are both heritable and reversible, and they can enable cells to switch dynamically between two or more phenotypic states, which commonly show differential responses to drug treatment. In fact, tumor cells in many cancer types have been observed to adopt reversible drug-tolerant or stem-like states in vitro, which enables the tumor to temporarily evade drug treatment and sets the stage for the evolution of more permanent resistance mechanisms^{7,8,9,10,11,12,13,14}. While these drug-tolerant states can exist independently of treatment^7,11,15, recent evidence indicates that common anti-cancer drugs can also induce or accelerate their adoption, a phenomenon we refer to as drug-induced plasticity^{8,11,12,15,16,17}. Drug induction is already well known in the context of genetic mutations, where both anti-cancer and anti-bacterial drugs have been observed to cause genomic instability or mutagenesis, referring to an elevated mutation rate occurring as a result of the stress imposed by the treatment^18,19,20,21.

Anti-cancer drug treatment has traditionally been administered under the “maximum tolerated dose” (MTD) paradigm, where the goal is to eradicate the tumor as quickly as possible and minimize the probability of spontaneous resistance-conferring mutations^22,23. Non-genetic cell plasticity fundamentally confounds the reasoning behind this strategy, especially if it is induced by the drug. First, if drug-tolerant states are adopted independently of the drug, it is likely that drug-tolerant cells will preexist treatment in large numbers, even if they form only a small fraction of the tumor bulk⁵. In addition, since non-genetic mechanisms usually operate much faster than genetic mutations^4,5,24, a non-negligible fraction of drug-sensitive cells will adopt drug tolerance under treatment. This means that it can be impossible to kill the tumor, no matter how large a dose is applied¹⁴. Second, if the drug induces tolerance adoption in a dose-dependent manner, applying larger doses with the aim of maximizing cell kill has the downside of promoting resistance evolution. This trade-off has been explored in several recent mathematical modeling works, which generally find that continuous low-dose or intermittent strategies outperform MTD treatment^{25,26,27,28,29,30}. However, these works usually focus on one potential model of drug-induced tolerance and some only consider the case of irreversible genetic resistance. In addition, many of these works only consider constant or intermittent strategies. Overall, there is a lack of generalized insights into how different potential forms of drug-induced tolerance influence optimal treatment decisions.

In this work, we seek a more systematic understanding of optimal dosing strategies under drug-induced cell plasticity. We seek both a qualitative understanding of the characteristics of optimal strategies and simple quantitative methods for computing them. We consider many different possible forms of drug-induced tolerance and explore how and why different forms lead to different optimal strategies. Our work differs from previous work in several important ways. First, whereas prior work generally assumes that tolerance is induced through elevated transitions from drug-sensitivity to drug-tolerance, it is also biologically plausible that the drug inhibits transitions from tolerance back to sensitivity, or that it affects both transitions simultaneously. For example, hypermethylation in the promoter regions of tumor suppressor genes has been associated with gene silencing and drug resistance in many cancers^1,4,31,32. When a cell divides, existing methylation patterns are preserved by the maintenance methyltransferase DNMT1, while de novo methylation is carried out by the methyltransferases DNMT3A/3B^33,34. If a drug-tolerant state is conferred via methylation, drug-induced upregulation of DNMT3A/3B will induce the adoption of the tolerant state, while drug-induced upregulation of DNMT1 will inhibit transitions out of it. The notion that the drug may not only elevate transitions from sensitivity to tolerance is further supported by experimental evidence indicating that the drug can influence all possible phenotypic transitions simultaneously^8,12,16. Second, whereas many prior works assume that the level of tolerance induction is linear as a function of the drug dose, we also consider the case where tolerance is induced uniformly in the presence of drug. These two forms of drug-induced tolerance were observed in a recent experimental work by Russo et al.¹⁷, and they emerge as limiting cases of a more general Michaelis-Menten form of tolerance induction (Methods). Third, while most previous work is computational in nature, we establish mathematical results which provide generalized insights that apply across all biologically relevant parameter regimes. Finally, whereas some previous works focus on irreversible transitions from drug-sensitivity to drug-resistance, we study the case of reversible phenotypic switching between sensitivity and tolerance. Our results are compared with previous work in more detail in Supplementary Note 1.

Results

Mathematical model of drug-induced tolerance enables derivation of optimal dosing strategies

We propose a mathematical model of a tumor undergoing anti-cancer drug treatment, in which cells are able to transition between two cell states, a drug-sensitive (type-0) and a drug-tolerant (type-1) state. Cells of type-i divide at rate b_i and die at rate d_i, with net division rate λ_i = b_i − d_i. Each drug-sensitive cell is assumed to transition to the drug-tolerant state at rate μ, and each tolerant cell transitions back to the sensitive state at rate ν (Fig. 1a). To model the effect of the drug on cell proliferation, we assume that the sensitive cell death rate d₀(c) increases with the drug dose c (Fig. 1b). Equivalently, it can be assumed that the drug decreases the division rate of sensitive cells (Methods). To model drug-induced tolerance, the transition rates μ(c) and ν(c) between sensitivity and tolerance are assumed to depend on the drug dose c, either in a linear or uniform fashion (Fig. 1c, d). A detailed description of the mathematical model is provided in the Methods section.

Fig. 1: Mathematical model of drug-induced tolerance.

a In the mathematical model, cells transition between two states, a drug-sensitive (type-0) state and a drug-tolerant (type-1) state. Sensitive cells divide at rate b₀, die at rate d₀ and transition to tolerance at rate μ. Tolerant cells divide at rate b₁, die at rate d₁ and transition to sensitivity at rate ν. The sensitive cell death rate d₀ and the transition rates μ and ν depend on the drug dose c. b The sensitive cell death rate follows a Michaelis-Menten equation of the form ${d}_{0}(c)={d}_{0}+({d}_{\max }-{d}_{0})c/(c+1)$, where d₀ is the death rate in the absence of drug and ${d}_{\max }$ is the saturation death rate under an arbitrarily large drug dose. The dose c is normalized to the EC₅₀ dose for d₀, meaning that the drug has half the maximal effect at dose c = 1. c The transition rate μ(c) from drug-sensitivity to drug-tolerance is assumed either a linearly increasing function of c or to be uniformly elevated in the presence of drug. These two forms of drug-induced tolerance were observed in a recent study by Russo et al.¹⁷ and they also emerge as limiting cases of more general Michaelis-Menten dynamics. d The transition rate ν(c) from drug-tolerance to drug-sensitivity is similarly assumed either a linearly decreasing function of c or uniformly inhibited in the presence of drug.

The time evolution of the number of drug-sensitive (type-0) and drug-tolerant (type-1) cells can be described by the differential equations

$$\begin{array}{rcl}\dfrac{d{n}_{0}}{dt}&=&({\lambda }_{0}(c)-\mu (c)){n}_{0}+\nu (c){n}_{1},\\ \dfrac{d{n}_{1}}{dt}&=&({\lambda }_{1}-\nu (c)){n}_{1}+\mu (c){n}_{0},\end{array}$$

where the drug dose is a function of time, c(t) (Methods). The proportion of drug-sensitive cells over time, f₀(t) = n₀(t)/(n₀(t) + n₁(t)), then follows the differential equation

$$\frac{d{f}_{0}}{dt}=\left({\lambda }_{1}-{\lambda }_{0}(c)\right){f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)\right){f}_{0}+\nu (c).$$

If a constant dose is applied, c(t) = c for all t ≥ 0, the tumor eventually grows exponentially at a fixed rate σ(c) given by

$$\sigma (c)=\frac{1}{2}\left({\lambda }_{0}(c)-\mu (c)+{\lambda }_{1}-\nu (c)+\sqrt{{({\lambda }_{0}(c)-\mu (c)-{\lambda }_{1}+\nu (c))}^{2}+4\mu\, (c)\nu (c)}\right).$$

(1)

Moreover, the intratumor composition eventually reaches an equilibrium, where the proportion of sensitive cells in the population becomes fixed at

$${\bar{f}}_{0}(c)=\frac{\nu (c)}{\sigma (c)-{\lambda }_{0}(c)+\mu (c)+\nu (c)}.$$

(2)

Thus, under a constant dose c, it is possible to maintain the tumor at a stable intratumor composition, which leads to a fixed exponential growth rate σ(c) (Methods).

We consider a dosing strategy (c(t))_{t∈[0, T]} optimal if it minimizes the total number of cells, n₀(T) + n₁(T), at the end of a finite time horizon [0, T]. When analyzing which dosing strategy is optimal in the long run, we also consider an infinite-horizon problem where the goal is to minimize the long-run average tumor growth rate (Methods). We can show that to determine the optimal strategy, it is sufficient to consider its effect on the proportion of sensitive cells f₀(t) over time, as opposed to its effect on the individual subpopulation counts n₀(t) and n₁(t) (Methods). This enables us to reduce an optimal control problem involving two state variables n₀(t) and n₁(t) to a single-state-variable problem, which simplifies our computational algorithms and mathematical analysis. In line with this insight, we use the proportion of sensitive cells over time to demonstrate the effects of optimal dosing strategies on tumor evolution in the following sections.

Constant low dosing is optimal in the long run under linear induction of tolerance

We now investigate optimal dosing strategies under linear induction of tolerance. We consider three cases, depicted in Fig. 2a, depending on whether the drug only affects transitions from sensitivity to tolerance (Case I), it only affects transitions from tolerance to sensitivity (Case II), or it affects both transitions simultaneously (Case III). In Fig. 2b, we show the optimal strategy for each case, computed using our own implementation of the forward–backward sweep method in MATLAB (Methods). For Case I, the optimal strategy applies a constant low dose c* from the beginning. For Case II, the optimal strategy starts with the maximum dose ${c}_{\max }$, and then gradually decreases it to a constant low dose c*. In this case, higher doses can be used at the onset of treatment to reduce the predominantly sensitive tumor more quickly, without the downside of promoting transitions from sensitivity to tolerance. For Case III, the optimal strategy has characteristics of both previous cases, as it does apply larger doses in the beginning, but it does not start with the maximum dose.

Fig. 2: Optimal dosing under linear induction of tolerance.

a In our investigation of linear induction of tolerance, we consider three cases. In Case I, the drug only increases the transition rate from sensitivity to tolerance (μ(c) = μ_0;+ kc and ν(c) = ν₀), in Case II, the drug only decreases the transition rate from tolerance to sensitivity (μ(c) = μ₀ and ν(c) = ν₀ − mc), and in Case III, the drug does both (μ(c) = μ₀ + kc and ν(c) = ν₀ − mc). b Optimal dosing strategies under linear induction of tolerance computed using the forward-backward sweep method. In all cases, the optimal dosing strategy involves both a transient phase and an equilibrium phase. During the equilibrium phase, a constant dose c* is applied, which minimizes the long-run tumor growth rate σ(c) under a constant dose c, as indicated by the dotted lines. The grayed-out part of each dosing strategy is a boundary effect which arises due to the optimization goal of minimizing the tumor size at a specific time T, with no regard for what is best in the long run. If we were to extend the treatment horizon beyond time T, the period applying the constant dose c* would be extended (Supplementary Fig. 1). We therefore ignore this boundary effect and focus on what is optimal over a longer time horizon. c Proportion of sensitive cells f₀(t) over time under the optimal dosing strategy. The constant equilibrium dose c* maintains the tumor at a fixed proportion between sensitive and tolerant cells. d Tumor size evolution under the optimal strategy compared with the tumor size evolution under the constant-dose strategy applying c* throughout. For Case I, the two strategies are the same. For Cases II/III, the optimal strategy results in 85.6%/55.4% greater tumor reduction during the initial stages of treatment, and 59.4%/24.2% greater reduction long-term. e If the proportion of sensitive cells f₀(0) at the start of treatment is changed, only the transient phase of the optimal treatment is affected, whereas the optimal equilibrium dose c* remains the same. For Case I, the transient phase is independent of the initial condition.

By combining these results with mathematical analysis, we can derive several general treatment insights. First, the optimal dosing strategy always involves both a transient phase and an equilibrium phase. During the equilibrium phase, a constant dose c* is applied, which maintains the tumor at a fixed proportion between sensitive and tolerant cells (Fig. 2(c)). This is true irrespective of the model parameter values (Supplementary Note 1). To demonstrate the importance of distinguishing these two treatment phases, we compare in Fig. 2d the optimal strategy with the constant-dose strategy applying the equilibrium dose c* throughout. Eventually, the tumor reduces at the same exponential rate under both strategies. However, during the transient phase, the optimal strategy reduces the tumor much faster for Cases II and III, and this difference in tumor size is maintained throughout the course of treatment.

Second, we can prove mathematically that the optimal equilibrium dose c* minimizes the long-run tumor growth rate σ(c) under a constant dose c (Supplementary Note 1). This means that the explicit expression (1) for σ(c) has clinical value on its own, as it can be used to address questions concerning ultimate treatment success. For example, if for a particular cancer type and a particular drug, we are interested in knowing whether there exists any dosing schedule capable of driving long-run tumor reduction (however complicated), we simply need to check whether there exists a constant dose c such that σ(c) < 0. We also note that since the equilibrium dose c* is determined by long-run tumor growth dynamics, it is not affected by changes in the initial proportion between sensitive and tolerant cells (Fig. 2e).

Third, we can say that the goal of the transient phase is to steer the tumor to a desired composition, and the goal of the equilibrium phase is to maintain it at that composition. We can show that during the transient phase, once the proportion of sensitive cells reaches some value f₀, the optimal dose c to give at that moment must maximize

$$\frac{\sigma ({c}^{* })-u(c,{f}_{0})}{-{f}^{{\prime} }_{{0}}},$$

(3)

where u(c, f₀) = (λ₀(c) − λ₁)f₀ + λ₁ is the instantaneous tumor growth rate (Supplementary Note 1). Applying the maximum dose ${c}_{\max }$ throughout the transient phase would kill the sensitive cells as quickly as possible, which would lead to the lowest possible growth rate u(c, f₀) and thus the largest possible numerator in (3). However, the maximum dose would simultaneously induce a large number of sensitive cells to adopt tolerance, which would lead to a large reduction $-{f}^{{\prime} }_{{0}}$ in the proportion of sensitive cells in the denominator of (3). Overall, expression (3) describes precisely how the optimal strategy must balance the trade-off between sensitive cell kill and tolerance induction as it steers the tumor to the optimal equilibrium composition.

Finally, our mathematical analysis suggests a simple and efficient method to compute the optimal dosing strategy, which simultaneously provides an insight into how the doses are selected. First, the optimal equilibrium dose c* can be derived by minimizing expression (1), and the associated equilibrium proportion of sensitive cells ${\bar{f}}_{0}({c}^{* })$ can be computed using expression (2). Then, the transient phase of the optimal strategy can be computed by iteratively determining the maximizing dose c in (3) and updating the value of f₀, as visualized in Supplementary Fig. 2a. In Supplementary Fig. 2b, we confirm that the transient phase computed using this approach agrees with the transient phase shown in Fig. 2 (obtained using the forward-backward sweep method).

Long treatment breaks significantly deteriorate treatment efficacy

Under linear induction of tolerance, it is optimal in the long run to expose the tumor continuously to the drug at a majority drug-tolerant composition. We are next interested in investigating how the optimal treatment compares with pulsed treatments, where the tumor is taken periodically off drug to allow tolerant cells to revert back to sensitivity. This comparison is made using the long-run average growth rate of the tumor (Methods).

We consider a pulsed schedule as a triple (c, t_cycle, φ) encoding the drug dose c, the duration of each treatment cycle t_cycle, and the proportion of time φ the drug is applied during each cycle. A constant schedule is considered a pulsed schedule with φ = 1. It is relatively simple to show that in our model, for any pulsed schedule (c, t_cycle, φ₁), there exists a schedule (c, t_cycle/2, φ₂) with half the cycle length which yields the same or lower long-run growth rate (Supplementary Note 1). Therefore, when investigating pulsed schedules, it is sufficient to consider idealized schedules with an arbitrarily short cycle length. However, it is also simple to show that once the cycle length has become sufficiently small, the long-run average growth rate becomes effectively invariant to the cycle length (Supplementary Note 1). Thus, even if the best pulsed schedule involves switching arbitrarily fast between drug and no drug, a comparable outcome can be obtained using a much longer cycle time. This is an important insight, as it means that the model-recommended pulsed schedule is more clinically feasible.

For a given pulsed schedule (c, φ) with an arbitrarily short cycle length, the long-run average growth rate of the tumor can be computed using the relatively simple formula (24). In Fig. 3a, we show the long-run growth rate for a wide range of possible pulsed schedules (c, φ) under Cases I, II and III. The red lines show for each proportion φ of drug exposure the optimal dose c*(φ) to administer, and the red dots indicate the optimal schedules. As expected, a constant low dose is optimal in the long run. Figure 3b, c show that for all three cases, the optimal schedule is quite sensitive to the duration of drug exposure. Instituting treatment breaks of 20–30% or longer results in a significantly larger tumor growth rate and a larger average dose being required to achieve that growth rate. From a clinical perspective, understanding to what extent the optimal strategy can be perturbed to insert treatment breaks can be useful for reducing potential side effects of continuous drug exposure. Plots like those in Fig. 3 could help a clinician understand the implications of instituting treatment breaks of a certain length, and to determine which dose is best for that length.

Fig. 3: Continuous and pulsed schedules under linear induction of tolerance.

a Long-run average growth rate of the tumor under a range of possible pulsed schedules of the form (c, φ), where c is the drug dose applied and φ is the proportion of drug exposure. The red curves show for each exposure proportion φ the optimal dose c*(φ) to give. The red dots show the optimal pulsed schedules, which are continuous for Cases I, II and III, consistent with our results in Fig. 2. b Long-run average growth rate under the optimal dose c*(φ), shown as a function of the proportion φ of drug exposure. c Average dose applied during each treatment cycle under the optimal dose c*(φ), shown as a function of the proportion φ of drug exposure, normalized to the average dose under the optimal schedule.

Pulsed maximum dosing is optimal in the long run under uniform induction of tolerance

We now consider the case where drug tolerance is induced in a uniform fashion, meaning that the drug effect is constant whenever the drug is present. In this case, the forward–backward sweep method does not converge to a unique solution, due to the discontinuity of the transition rate functions μ(c) and ν(c). We therefore rely on mathematical analysis.

First, we can show that in the long run, it is optimal to apply a pulsed schedule alternating arbitrarily quickly between the maximum dose ${c}_{\max }$ and no dose (Supplementary Note 1). To verify this result computationally, we compare in Fig. 4b the long-run average tumor growth rate over a wide range of possible pulsed schedules (c, φ). For each given proportion of drug exposure φ, it is optimal to apply the maximum dose ${c}_{\max }$ whenever the drug is present, as indicated by the red lines. For Case I, it is optimal to apply the maximum dose continuously, and for Cases II and III, it is optimal to alternate between the maximum dose and no dose, as indicated by the red dots. Thus, whereas low continuous dosing is optimal in the long run for linear induction of tolerance, for uniform induction, it is optimal to apply the maximum dose either continuously or intermittently. This result is intuitive since under uniform induction, once any drug dose is applied, there is no downside in applying a larger dose.

Fig. 4: Optimal dosing under uniform induction of tolerance.

a In our investigation of uniform induction of tolerance, we consider three cases. In Case I, the drug only increases the transition rate from sensitivity to tolerance (μ(0) = μ₀ and $\mu (c)={\mu }_{\max }$ for c > 0, ν(c) = ν₀ for c ≥ 0), in Case II, the drug only decreases the transition rate from tolerance to sensitivity (μ(c) = μ₀ for c ≥ 0, ν(0) = ν₀ and $\nu (c)={\nu }_{\min }$ for c > 0), and in Case III, the drug does both (μ(0) = μ₀ and $\mu (c)={\mu }_{\max }$ for c > 0, ν(0) = ν₀ and $\nu (c)={\nu }_{\min }$ for c > 0). b The heatmaps show the long-run average tumor growth rate for a range of possible pulsed schedules of the form (c, φ) for Cases I, II and III. The red curves show each for exposure proportion φ the optimal dose c*(φ) to give, and the red dots show the optimal pulsed schedules. For each possible exposure proportion φ, it is optimal to apply the maximum dose ${c}_{\max }$. For Case I, where the drug only affects transitions from sensitivity to tolerance, it is optimal to apply the maximum dose continuously. For Cases II and III, it is optimal to alternate between the maximum dose and no dose. c Under uniform induction of tolerance, we can prove that during the transient phase, it is optimal to apply the maximum dose throughout, and during the equilibrium phase, it is optimal to apply a pulsed schedule $({c}_{\max },\varphi )$ alternating between the maximum allowed dose ${c}_{\max }$ and no dose (possibly with φ = 1). Examples of optimal strategies are shown for Cases I, II and III. For Case I, where the drug only increases transitions from sensitivity to tolerance, it is optimal to apply the maximum dose continuously throughout the entire treatment period. d Proportion of sensitive cells f₀(t) over time under the optimal dosing strategy. Same as for linear induction of tolerance, the optimal dosing strategy maintains the tumor at a fixed proportion between sensitive and tolerant cells in the long run.

Second, we can show that the optimal transient strategy is to apply the maximum dose ${c}_{\max }$ continuously under Cases I, II and III (Supplementary Note 1). This enables us to derive optimal dosing strategies for all cases under uniform induction of tolerance (Fig. 4c). For Case I, it is optimal to apply the maximum dose ${c}_{\max }$ continuously from the beginning. This implies that drug-induced tolerance does not necessarily call for low or intermittent dosing. For Cases II and III, it is optimal to start with the maximum dose and then alternate between the maximum dose and no dose. For all cases, the optimal strategy eventually maintains the tumor at a fixed composition between sensitive and tolerant cells (Fig. 4d). Thus, same as for the case of linear induction, the optimal strategy involves a transient and an equilibrium phase, where the goal of the transient phase is to steer the tumor to a desired composition, and the goal of the equilibrium phase is to maintain it at that composition.

In this section, we have referred to a strategy alternating arbitrarily quickly between the maximum dose and no dose as the optimal long-run strategy. Since this is an idealized schedule, it is more precise to say that there exists a sequence of pulsed strategies which achieve the optimal long-run growth rate as t_cycle approaches zero. However, same as for the case of linear induction, the cycle length can be extended significantly without affecting the long-run average growth rate of the tumor (Supplementary Note 1).

Integration with in vitro patient tumor data produces individualized clinical insights

Linear and uniform induction of tolerance represent extreme cases where the level of tolerance induction either increases gradually with the dose or is insensitive to the dose. In the former case, lower doses are generally preferred, whereas in the latter case, the maximum dose should be applied whenever the drug is present. If a clinician understands which case applies better to a particular cancer type, for example, based on experience with prior patients, these general insights can help guide clinical decision-making. If the biological mechanism of drug-induced tolerance is furthermore known, it may become possible to distinguish between Cases I, II and III. For linear induction, this can help decide whether to apply low or high doses during the initial stages of treatment, and for uniform induction, it can help decide whether to apply the maximum dose continuously or intermittently in the long run.

Our approach can yield even more specific clinical insights when integrated with experiments involving in vitro models of patient tumors. Technologies to derive clinically relevant cell culture models from patient tumor samples are advancing rapidly^{35,36,37,38,39}, and their development has recently been put into focus by the FDA Modernization Act 2.0, which seeks to replace animal models in drug safety and effectiveness testing by in vitro human cell models and in silico computational modeling^40,41,42. As a case study, we now apply our insights to in vitro experiments from Russo et al.¹⁷, where the authors treated two colorectal cancer cell lines with increasing doses of anti-cancer agents. They found that when BRAF V600-E mutated WiDr colorectal cancer cells were treated with a combination of dabrafenib and cetuximab, the drugs linearly induced transitions from a drug-sensitive to a drug-persistent state (Case I under linear induction). For RAS/RAF wild-type DiFi cells treated with cetuximab, the drug uniformly induced the same transitions (Case I under uniform induction). We assume that the drug does not affect transitions from persistence to sensitivity, since no such effect is indicated in ref. ¹⁷.

According to our analysis in the previous section, we already know that for the DiFi cells, it is optimal to apply the maximum dose continuously. We therefore focus on the WiDr cells and compare the optimal strategy with dosing regimens considered in ref. ¹⁷. For the purposes of this analysis, we adopt the model of drug effect on cell proliferation from ref. ¹⁷ (Methods). As expected from our analysis in the section on linear induction of tolerance, the optimal strategy over a treatment horizon of T = 100 days applies a constant dose from the beginning (Fig. 5a). The optimal dose is even smaller here than for the examples considered in Fig. 2 or c* = 0.85 as a proportion of the ${{\rm{EC}}}_{50}^{d}$ dose (in absolute terms, the ${{\rm{EC}}}_{50}^{d}$ dose is 2.3779 × 10⁻⁷M and the optimal dose is 2 × 10⁻⁷M). One reason that the optimal dose is lower than the ${{\rm{EC}}}_{50}^{d}$ dose in this case is that the dose at which the net sensitive cell division rate λ₀(c) becomes negative is much smaller than the ${{\rm{EC}}}_{50}^{d}$ dose, meaning that the tumor reduces in size even at c* = 0.85 (Methods).

Fig. 5: Optimal dosing for an in vitro colorectal cancer system.

a For BRAF V600-E mutated WiDr cells, Russo et al.¹⁷ observed linearly increasing transitions from sensitivity to tolerance under dabrafenib + cetuximab treatment. Under a sufficiently long time horizon, the optimal dosing strategy applies a low constant dose equal to 85% of the ${{\rm{EC}}}_{50}^{d}$ dose. b In ref. ¹⁷, the authors consider both a constant strategy and a linearly increasing strategy with average dose $\bar{c}=4.2$ (relative to the ${{\rm{EC}}}_{50}^{d}$ dose) using a time horizon of T = 10 days. The optimal strategy over T = 10 days applies small doses during most of the period and increases the dose towards the end of the horizon. c The linearly increasing schedule with average dose $\bar{c}=4.2$ outperforms the constant-dose schedule with the same average dose in terms of tumor reduction over T = 10 days, which is consistent with the results of ref. ¹⁷. However, the optimal strategy over T = 10 days achieves a 19% greater tumor reduction while applying half as much dose. Moreover, the optimal long-run constant dose c* = 0.85 achieves similar tumor reduction to the linear strategy at less than 20% of the cumulative dose. d Out of the four schedules considered in part b, the optimal long-run constant dose c* = 0.85 induces the fewest sensitive cells to adopt persistence over T = 10 days.

In ref. ¹⁷, the authors considered a shorter time horizon of T = 10 days, and they compared a constant dose schedule with c = 4.2, relative to the ${{\rm{EC}}}_{50}^{d}$ dose, with a linearly increasing dose schedule involving the same average dose $\bar{c}=4.2$ over the horizon (Fig. 5b). They observed that the linearly increasing schedule significantly outperformed the constant-dose schedule, which is confirmed by our analysis (Fig. 5c). However, we note that the considerable preference for the linear schedule is due to the constant-dose schedule with c = 4.2 being significantly suboptimal. For example, the long-run optimal dose c* = 0.85 leads to a similar tumor decay over T = 10 days as the linear schedule with $\bar{c}=4.2$ (Fig. 5c). The optimal short-term strategy over T = 10 days applies small doses for most of the period, starting at c = 0.90, before increasing the dose at the very end (Fig. 5b). The optimal short-term strategy achieves a 19% greater tumor reduction than the linear strategy, while applying less than half as much cumulative dose, or $\bar{c}=2.0$ on average.

Here, we have evaluated the treatment schedules in terms of the tumor size at the end of the treatment horizon. In ref. ¹⁷, the authors considered a different metric of treatment success, the number of persister cells induced under treatment, assuming no persister cells at the beginning of treatment. Even under this metric, the optimal short-term strategy outperforms the linearly increasing schedule, and in fact, the optimal long-run dose c* = 0.85 performs the best (Fig. 5d). From one perspective, this is not surprising, since if the goal is to minimize the number of sensitive cells that the drug induces to adopt persistence, it is optimal to give no drug and thus induce no sensitive cells to adopt persistence. On the other hand, the fact that the optimal long-run dose induces fewer persister cells than the other three schedules in the short term is precisely why it is better in the long run.

In summary, the constant-dose strategy with c = 0.85 is both successful at restricting persister evolution in the short term and optimally controlling the tumor in the long term. In addition, it applies around an 80% lower cumulative dose than the linear schedule with average dose $\bar{c}=4.2$. These results show how applying our approach to experiments involving in vitro models of patient tumors can yield individualized clinical insights. Determining the optimal dose, understanding how it affects the tumor in the short and long term, and comparing it with other potential dosing strategies would be difficult without the mathematical model.

Discussion

In this work, we have investigated optimal dosing of anti-cancer treatment under drug-induced plasticity. We found that the optimal strategy always involves both a transient phase and an equilibrium phase, where during the equilibrium phase, the tumor is maintained at a fixed intratumor composition that minimizes its long-run growth rate. Under linear induction of tolerance, the optimal equilibrium strategy is to apply a low constant dose, while under uniform induction of tolerance, it is optimal to apply the maximum dose either continuously or intermittently. We proved this mathematically and provided simple methods for computing the optimal strategies. We also showed that during the transient phase, the optimal strategy steers the tumor to the desired long-run composition while precisely balancing the trade-off between cell kill and tolerance induction. We discussed how these general insights can help guide clinical decision-making, and we showed how more specific insights can be obtained by integrating our methodology with experiments involving in vitro models of patient tumors.

We view this work as an initial step towards a more complete understanding of optimal dosing under drug-induced plasticity. We have left many important questions unaddressed which serve as a guide for future work. First, we have only considered the extreme cases where drug tolerance is induced in a linear or uniform fashion. If the transition rate functions μ(c) and ν(c) follow more general Michaelis-Menten or Hill dynamics⁴³, it may be difficult to apply optimal control analysis, but the optimal pulsed strategy can always be determined using the explicit expression (24). Second, while we have assumed that cells transition directly between a drug-sensitive and a drug-tolerant state, these transitions may occur through intermediate states or in a more continuous fashion. In addition, there is evidence that prolonged drug exposure can induce epigenetic reprogramming of tolerant cells and eventually lead to stable drug resistance^7,11. Tolerant cells can furthermore form a reservoir for the evolution of eventual genetic resistance^9,10,17. This suggests the need to include more cell states in the model, which is easily accommodated within our framework and we plan to do in future work. Alternatively, the cell state can be treated as a continuous variable, representing gradual evolution towards resistance^{44,45,46,47,48}. Nevertheless, we believe that our analysis of the dynamics between sensitive and tolerant cells yields valuable insights that will be useful for understanding the more complex dynamics. Third, we have considered an exponential growth model without competitive or cooperative dynamics. If tumor growth is constrained by a carrying capacity, our insights will continue to hold whenever it is possible to kill the tumor, since then the carrying capacity will not significantly affect the dynamics. However, if recurrence cannot be avoided, it may become optimal to keep the tumor close to the carrying capacity, in order to avoid releasing tolerant cells from competition with sensitive cells^25,27,49. This will be important to explore in future work. Finally, we have equated drug dose with drug concentration throughout and ignored the effect of pharmacokinetics, as well as unwanted toxic side effects of the drug. Confirming the clinical applicability of our approach would require an investigation of these dynamics, particularly for the case of uniform tolerance induction, where the dynamics are significantly different depending on whether the drug is present or not.

Taking full advantage of our approach for a specific cancer type or a specific patient requires understanding the quantitative dynamics of drug induction for that specific case. Russo et al.¹⁷ have suggested a Bayesian approach for inferring the transition rate function μ(c) from sensitivity to tolerance using in vitro tumor bulk data, but their work only distinguishes between the extreme cases of linear and uniform induction of tolerance. In a recent work, we have suggested a maximum likelihood framework for inferring more general dose-response dynamics for μ(c) based on the Hill equation⁵⁰. However, as we have discussed, it is also plausible that the drug inhibits the transition rate ν(c) from tolerance back to sensitivity, or that the drug simultaneously affects both rates. In general, the present work shows that the optimal strategy varies significantly depending on the exact dynamics of tolerance induction. This suggests the need to develop integrated experimental and mathematical tools capable of jointly inferring d₀(c), μ(c) and ν(c) from experimental data, which may require data on the phenotypic composition of the tumor⁵¹. Ultimately, these integrated tools can help usher in a new era of precision medicine, where the dynamics of drug induction are determined and dosing strategies are optimized on an individual patient basis^52,53.

Methods

Model of cell proliferation and phenotypic switching

Our baseline model is a multi-type branching process model in continuous time⁵⁴. In the model, cells switch stochastically between two distinct cell states, a drug-sensitive (type-0) and a drug-tolerant (type-1) state. In the absence of drug, a type-0 cell divides into two type-0 cells at rate b₀, it dies at rate d_0, and it transitions to the tolerant type-1 state at rate μ > 0. More precisely, each type-0 cell waits an exponentially distributed amount of time with rate a₀ ≔ b₀ + d₀ + μ before it either divides with probability b₀/a₀, dies with probability d₀/a₀, or transitions to type-1 with probability μ/a₀. A type-1 cell divides at rate b₁, it dies at rate d₁, and it reverts to type-0 at rate ν > 0 (Fig. 1a). The net birth rates of the two types are denoted by λ₀ ≔ b₁ − d₁ and λ₁ ≔ b₁ − d₁.

Drug effect on cell proliferation

To model the effect of the anti-cancer drug on cell proliferation, we assume that the sensitive cell death rate d₀ is an increasing function of the current drug dose c. For simplicity, we assume that the drug-tolerant cells are unaffected by the drug. The specific functional form for d₀ is the Michaelis-Menten function, which is commonly used for this purpose:

$${d}_{0}(c)={d}_{0}+({d}_{\max }-{d}_{0})\cdot \frac{c}{c+{{\rm{EC}}}_{50}^{d}}={d}_{0}+\Delta {d}_{0}\cdot \frac{c}{c+{{\rm{EC}}}_{50}^{d}}.$$

(4)

Here, d₀ is the sensitive cell death rate in the absence of drug, ${d}_{\max }$ is the saturation death rate under an arbitrarily large drug dose, $\Delta {d}_{0}:= {d}_{\max }-{d}_{0}\ge 0$ is their difference, and ${{\rm{EC}}}_{50}^{d}$ is the dose at which the drug has half the maximal effect on sensitive cell proliferation. Note that we write simply d₀ instead of d₀(0) to signify the death rate in the absence of drug. The function in (4) is concave, but it is worth noting that if the the drug dose is viewed on a log scale, then (4) becomes an S-shaped dose-response curve which is characteristic of viability curves in the biological literature (Fig. 6).

Fig. 6: Drug effect on sensitive cell death rate d₀.

The Michaelis-Menten function (4) describing the drug effect on cell proliferation is concave in the drug dose c (left panel). When the dose c is viewed on a log scale, the function becomes takes on an S-shape which is characteristic of viability curves in the biological literature (right panel).

Expression (4) can be rewritten as

$${d}_{0}(c)={d}_{0}+\Delta {d}_{0}\cdot \frac{(c/{{\rm{EC}}}_{50}^{d})}{(c/{{\rm{EC}}}_{50}^{d})+1}.$$

Thus, if we measure the drug dose as a proportion of the ${{\rm{EC}}}_{50}^{d}$ dose, we obtain the simpler expression

$${d}_{0}(c)={d}_{0}+\Delta {d}_{0}\cdot \frac{c}{c+1}.$$

(5)

We will usually adopt this scaling of the drug dose and use the simpler form for d₀(c). Similarly, the net birth rate of sensitive cells is given by

$${\lambda }_{0}(c)={b}_{0}-{d}_{0}(c)={\lambda }_{0}-\Delta {d}_{0}\cdot \frac{c}{c+1},$$

(6)

where we simply write λ₀ instead of λ₀(0).

While we have assumed in the preceding discussion that the drug increases the death rate of sensitive cells (cytotoxic drug), we can equivalently assume that the drug decreases their division rate (cytostatic drug). This is because our results depend on the division rate and death rate of sensitive cells only through the net birth rate λ₀(c). If we assume that a cytostatic drug influences the division rate as follows:

$${b}_{0}(c)={b}_{0}-\Delta {b}_{0}\cdot \frac{c}{c+1},$$

the net division rate becomes

$${\lambda }_{0}(c)={b}_{0}(c)-{d}_{0}={\lambda }_{0}-\Delta {b}_{0}\cdot \frac{c}{c+1},$$

which has the same form as expression (6).

Drug effect on switching rates

To model the possibility of drug-induced tolerance, we assume that in the presence of the anti-cancer drug, one or both of the transition rates μ and ν change with the drug dose c. If we focus first on the transition rate μ from sensitivity to tolerance, it is natural to assume that it follows a Michaelis-Menten function of the form (4):

$$\mu (c)={\mu }_{0}+({\mu }_{\max }-{\mu }_{0})\cdot \frac{c}{c+{{\rm{EC}}}_{50}^{\mu }}={\mu }_{0}+\Delta \mu \cdot \frac{c}{c+{{\rm{EC}}}_{50}^{\mu }},$$

(7)

where $\Delta \mu := {\mu }_{\max }-{\mu }_{0}\ge 0$. The dose at which the drug has half the maximal effect on μ, ${{\rm{EC}}}_{50}^{\mu }$, is in general distinct from ${{\rm{EC}}}_{50}^{d}$. Here, we will focus on the two extreme cases where ${{\rm{EC}}}_{50}^{\mu }\,\gg\, {{\rm{EC}}}_{50}^{d}$ and ${{\rm{EC}}}_{50}^{\mu }\,\ll\, {{\rm{EC}}}_{50}^{d}$. The case ${{\rm{EC}}}_{50}^{\mu }\,\gg\, {{\rm{EC}}}_{50}^{d}$ means that the drug induces tolerance at doses which are already very effective at killing the sensitive cells. In this case, if the drug dose c is similar in magnitude to ${{\rm{EC}}}_{50}^{d}$, then $c\,\ll\, {{\rm{EC}}}_{50}^{\mu }$ and by Taylor expansion:

$$\mu (c)={\mu }_{0}+\Delta \mu \cdot \frac{c/{{\rm{EC}}}_{50}^{\mu }}{c/{{\rm{EC}}}_{50}^{\mu }+1}\approx {\mu }_{0}+\frac{\Delta \mu }{{{\rm{EC}}}_{50}^{\mu }}\cdot c.$$

In other words, μ(c) can be treated as linearly increasing in c. The case ${{\rm{EC}}}_{50}^{\mu }\,\ll\, {{\rm{EC}}}_{50}^{d}$ means that the drug has fully induced tolerance at low doses which are not yet effective at killing the sensitive cells. In this case, if the drug dose c is similar in magnitude to ${{\rm{EC}}}_{50}^{d}$, then $c\,\gg\, {{\rm{EC}}}_{50}^{\mu }$ and by Taylor expansion for c > 0:

$$\mu (c)={\mu }_{0}+\Delta \mu \cdot \frac{1}{1+{{\rm{EC}}}_{50}^{\mu }/c}\approx {\mu }_{0}+\Delta \mu ={\mu }_{\max }.$$

In other words, μ is effectively constant in the presence of drug.

Based on the preceding, we assume the following two functional forms for μ:

• (1) μ(c) = μ₀ + kc, where k ≥ 0 (tolerance is linearly induced).

• (2) μ(0) = μ₀ and $\mu (c)={\mu }_{\max }$ for c > 0 (tolerance is uniformly induced).

The reason we focus on these two extremes is twofold. First, these two exact forms were observed empirically in the recent investigation by Russo et al.¹⁷, which is the only work we know of which has attempted to quantify drug-induced tolerance in a dose-dependent manner using in vitro data. Second, due to their simplicity relative to the general case (7), we are able to establish rigorous mathematical results on optimal dosing strategies.

For the transition rate ν from tolerance back to sensitivity, the corresponding Michaelis-Menten function has the following form:

$$\nu (c)={\nu }_{0}+({\nu }_{\min }-{\nu }_{0})\cdot \frac{c}{c+{{\rm{EC}}}_{50}^{\nu }}={\nu }_{0}-\Delta \nu \cdot \frac{c}{c+{{\rm{EC}}}_{50}^{\nu }},$$

where $\Delta \nu := {\nu }_{0}-{\nu }_{\min }\ge 0$. We will also assume the following two functional forms for ν:

• (1) ν(c) = ν₀ − mc, where m ≥ 0.

• (2) ν(0) = ν₀ and $\nu (c)={\nu }_{\min }$ for c > 0.

By assumption, ν is a decreasing function of c, since we are modeling the case where the drug inhibits transitions from drug-tolerance to drug-sensitivity.

Throughout, we assume that μ₀ > 0 and ν₀ > 0, meaning that cells are able to transition between the two states even in the absence of drug.

System equations

Let Z₀(t) and Z₁(t) denote the number of sensitive and tolerant cells, respectively, at time t. Let ${n}_{0}(t):= {\mathbb{E}}[{Z}_{0}(t)]$ and ${n}_{1}(t):= {\mathbb{E}}[{Z}_{1}(t)]$ be the mean number of sensitive and tolerant cells at time t. These mean functions satisfy the differential equations

$$\begin{array}{rcl}\dfrac{d{n}_{0}}{dt}&=&({\lambda }_{0}(c)-\mu (c)){n}_{0}+\nu (c){n}_{1},\\ \dfrac{d{n}_{1}}{dt}&=&({\lambda }_{1}-\nu (c)){n}_{1}+\mu (c){n}_{0}.\end{array}$$

(8)

Here, the dose c should be viewed as a function of time, c(t). The total population size n(t) ≔ n₀(t) + n₁(t) obeys the differential equation

$$\frac{dn}{dt}={\lambda }_{0}(c){n}_{0}+{\lambda }_{1}{n}_{1}.$$

(9)

Let f₀(t) ≔ n₀(t)/n(t) and f₁(t) ≔ 1 − f₀(t) be the proportions of each cell type in the population at time t. Then

$$\begin{array}{lll}\dfrac{d{f}_{0}}{dt}&=&\dfrac{(d{n}_{0}/dt)\cdot n-{n}_{0}\cdot (dn/dt)}{{n}^{2}}\\ &=&\dfrac{\left(({\lambda }_{0}(c)-\mu (c)){n}_{0}+\nu (c){n}_{1}\right)n-{n}_{0}\left({\lambda }_{0}(c){n}_{0}+{\lambda }_{1}{n}_{1}\right)}{{n}^{2}}\\ &=&({\lambda }_{0}(c)-\mu (c)){f}_{0}+\nu (c)(1-{f}_{0})-{f}_{0}\left({\lambda }_{0}(c){f}_{0}+{\lambda }_{1}(1-{f}_{0})\right)\\ &=&\left({\lambda }_{1}-{\lambda }_{0}(c)\right){f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)\right){f}_{0}+\nu (c).\end{array}$$

(10)

Expression (9) can be rewritten as follows:

$$\frac{dn}{dt}=\left({\lambda }_{0}(c)\,{f}_{0}+{\lambda }_{1}(1-{f}_{0})\right)n=\left(({\lambda }_{0}(c)-{\lambda }_{1}){f}_{0}+{\lambda }_{1}\right)n.$$

(11)

Solving this differential equation, the mean tumor size at time T is given by the explicit expression

$$n(T)=n(0)\exp \left(\mathop{\int}\nolimits_{0}^{T}\left(({\lambda }_{0}(c)-{\lambda }_{1})\,{f}_{0}+{\lambda }_{1}\right)dt\right).$$

(12)

Expressions (11) and (12) indicate that the instantaneous growth rate of the population at time t is given by

$$u(c,{f}_{0}):= ({\lambda }_{0}(c)-{\lambda }_{1}){f}_{0}+{\lambda }_{1},$$

(13)

which will be an important quantity in our investigation of optimal treatment strategies.

Behavior under a constant dose

When a constant drug dose is applied, c(t) = c, all model parameters λ₀(c), λ₁, μ(c) and ν(c) are constant. In this case, the system equations (8) admit explicit solutions. To simplify the notation, we let λ₀, λ₁, μ and ν denote the constant parameters. We first note that the dynamics of the system can be encoded in the so-called infinitesimal generator

$${\bf{A}}:= \left[\begin{array}{cc}{\lambda }_{0}-\mu &\mu \\ \nu &{\lambda }_{1}-\nu ,\end{array}\right],$$

where the (i + 1, j + 1)-th element describes the net rate at which a cell of type-i produces a cell of type-j. The infinitesimal generator has distinct real eigenvalues ρ < σ given by

$$\begin{array}{rcl}\sigma &=&\dfrac{1}{2}\left(({\lambda }_{0}-\mu )+({\lambda }_{1}-\nu )+\sqrt{{\left(({\lambda }_{0}-\mu )-({\lambda }_{1}-\nu )\right)}^{2}+4\mu \nu }\right),\\ \rho &=&\dfrac{1}{2}\left(({\lambda }_{0}-\mu )+({\lambda }_{1}-\nu )-\sqrt{{\left(({\lambda }_{0}-\mu )-({\lambda }_{1}-\nu )\right)}^{2}+4\mu \nu }\right).\end{array}$$

Now define

$$\begin{array}{rcl}\delta &:= &\dfrac{({\lambda }_{0}-\mu )-\rho }{\nu },\\ \beta &:= &\dfrac{\sigma -({\lambda }_{0}-\mu )}{\nu }.\end{array}$$

If n₀(0) = n and n₁(0) = m, the mean number of cells of each type can be written explicitly as

$$\begin{array}{rcl}{n}_{0}(t)&=&\dfrac{n\delta +m}{\delta +\beta }{e}^{\sigma t}+\dfrac{n\beta -m}{\delta +\beta }{e}^{\rho t},\\ {n}_{1}(t)&=&\dfrac{\beta (n\delta +m)}{\delta +\beta }{e}^{\sigma t}-\dfrac{\delta (n\beta -m)}{\delta +\beta }{e}^{\rho t}.\end{array}$$

(14)

These expressions indicate two important aspects of the long-run dynamics. First, the tumor eventually grows (or decays) at exponential rate σ. Second, the intratumor composition eventually reaches an equilibrium where the proportion between type-1 and type-0 becomes the constant β. We denote the equilibrium proportion of the type-0 cells as

$${\bar{f}}_{0}:= \frac{1}{1+\beta }=\frac{\nu }{\sigma -{\lambda }_{0}+\mu +\nu }.$$

(15)

When we are considering a constant treatment, c(t) = c, we will write σ(c) and ${\bar{f}}_{0}(c)$ to explicitly denote the dependence on the dose c. For detailed derivations of the above formulas, we refer to Appendix A.2 of our previous work¹⁴.

Optimal control problem

To determine the optimal dosing strategy, we assume a finite treatment horizon [0, T] with T > 0. Let PC([0, T]) denote the set of all piecewise continuous functions on [0, T]. The space of allowable dosing strategies is

$${{\mathcal{C}}}_{T}:= \{c\in {\rm{PC}}([0,T]):c(t)\le {c}_{\max },\,\forall t\in [0,T]\},$$

where ${c}_{\max }$ is the maximum allowable instantaneous dose. For simplicity, when deriving optimal dosing strategies, we assume that the drug dose corresponds perfectly to the drug concentration reaching the tumor, thereby ignoring pharmacokinetic effects. Our main objective is to find the dosing strategy $c\in {{\mathcal{C}}}_{T}$ which minimizes the expected tumor size at the end of the treatment period. More specifically, we aim to solve the optimal control problem

$$\mathop{\inf }\limits_{c\in {{\mathcal{C}}}_{T}}n(T)=\mathop{\inf }\limits_{c\in {{\mathcal{C}}}_{T}}({n}_{0}(T)+{n}_{1}(T)).$$

By (12) and (13), it is equivalent to solve the problem

$$\mathop{\inf }\limits_{c\in {{\mathcal{C}}}_{T}}\mathop{\int}\nolimits_{0}^{T}u(c,{f}_{0})dt,$$

(16)

where u(c, f₀) ≔ (λ₀(c) − λ₁)f₀ + λ₁ is the instantaneous growth rate of the population and f₀(t) is governed by the differential equation

$$\frac{d{f}_{0}}{dt}=\left({\lambda }_{1}-{\lambda }_{0}(c)\right){f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)\right){f}_{0}+\nu (c).$$

Note that under the assumption that c is piecewise continuous on [0, T], f₀ is piecewise continuously differentiable on [0, T]. Over a finite treatment horizon [0, T], the average growth rate of the population is given by

$${\bar{u}}_{T}(c)=\frac{1}{T}\mathop{\int}\nolimits_{0}^{T}u(c,{f}_{0})dt.$$

In our investigation of the optimal control problem (16), we will be interested in knowing which treatment is the best in the long run as T → ∞. Since the integral in (16) can be unbounded as T → ∞, we will also consider the long-run average growth rate

$${\bar{u}}_{\infty }(c)=\mathop{{\rm{lim}}\,{\rm{sup}}}\limits_{T\to \infty }{\bar{u}}_{T}(c)=\mathop{{\rm{lim}}\,{\rm{sup}}}\limits_{T\to \infty }\frac{1}{T}\mathop{\int}\nolimits_{0}^{T}u(c,{f}_{0})dt,$$

(17)

where $c:[0,\infty )\to [0,{c}_{\max }]$ is considered a treatment with an infinite time horizon.

Implementation of forward–backward sweep method

For the case of linear induction of tolerance, we use the forward–backward sweep method^55,56 to solve the optimal control problem (16). If we set

$$g(\,{f}_{0},c):= \left({\lambda }_{1}-{\lambda }_{0}(c)\right){f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)\right){f}_{0}+\nu (c),$$

then f₀(t) is governed by the differential equation df₀/dt = g(f₀, c). The so-called Hamiltonian for this problem is

$${\mathcal{H}}(\,{f}_{0},c,\gamma )=u(\,{f}_{0},c)+\gamma g(\,{f}_{0},c),$$

(18)

where γ = (γ(t))_{t∈[0, T]} is a Lagrangian multiplier function. By Pontryagin’s principle, the optimal solution ${c}^{* }={({c}^{* }(t))}_{t\in [0,T]}$ must satisfy

$${c}^{* }(t)={{\rm{argmin}}}_{c}{\mathcal{H}}({f}_{0},c,\gamma )$$

for each t ∈ [0, T], where ${f}_{0}={({f}_{0}(t))}_{t\in [0,T]}$ and (γ(t))_{t∈[0, T]} satisfy the differential equations

$$\begin{array}{rcl}\dfrac{\partial {\mathcal{H}}}{\partial \gamma }&=&{f}^{{\prime} }_{{0}},\\ \dfrac{\partial {\mathcal{H}}}{\partial {f}_{0}}&=&-{\gamma }^{{\prime} }.\end{array}$$

The boundary conditions are f₀(0) = α for some fixed α ∈ [0, 1] and γ(T) = 0. For our problem, the two differenial equations above become

$${f}^{{\prime} }_{{0}}=({\lambda }_{1}-{\lambda }_{0}(c))\,{f}_{0}^{2}-({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)){f}_{0}+\nu (c),\quad {f}_{0}(0)=\alpha ,$$

(19)

$$\begin{array}{l}{\gamma }^{{\prime} }=-\left(2({\lambda }_{1}-{\lambda }_{0}(c))\,{f}_{0}-({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c))\right)\\\quad\gamma -({\lambda }_{0}(c)-{\lambda }_{1}),\quad \gamma (T)=0.\end{array}$$

(20)

To compute the optimal solution, we apply the forward-backward sweep method, which is an iterative procedure that proceeds as follows:

• (1) We discretize time and consider a set of time points ${\mathcal{T}}=[0,{t}_{1},{t}_{2},\ldots ,T]$.

• (2) We make an initial guess for the optimal policy ${(c(t))}_{t\in {\mathcal{T}}}$. We simply use c(t) = 0 for all $t\in {\mathcal{T}}$.

• (3) We solve (19) using the initial condition f₀(0) = α, assuming the initial guess for ${(c(t))}_{t\in {\mathcal{T}}}$.

• (4) We solve (20) backwards in time using the initial condition γ(T) = 0, assuming the initial guess for ${(c(t))}_{t\in {\mathcal{T}}}$ and the trajectory for ${({f}_{0}(t))}_{t\in {\mathcal{T}}}$ from step (3).

• (5) Using the obtained trajectories for (f₀(t)) and (γ(t)), for each fixed $t\in {\mathcal{T}}$ we compute the optimal dose at time t as

  $${c}^{* }(t)={{\rm{argmax}}}_{c}{\mathcal{H}}(\,{f}_{0},c,\gamma ),$$

  which boils down to solving

  $$\frac{\partial {\mathcal{H}}}{\partial c}=0.$$

  (21)

  For our problem, this condition can be written as a second-degree polynomial equation in c. Since

  $$\begin{array}{lll}\dfrac{\partial u}{\partial c}\,=\,{f}_{0}\dfrac{\partial {\lambda }_{0}}{\partial c},\\ \dfrac{\partial g}{\partial c}\;=\;-{f}_{0}^{2}\dfrac{\partial {\lambda }_{0}}{\partial c}+{f}_{0}\dfrac{\partial {\lambda }_{0}}{\partial c}-{f}_{0}\dfrac{\partial \mu }{\partial c}-{f}_{0}\dfrac{\partial \nu }{\partial c}+\dfrac{\partial \nu }{\partial c},\end{array}$$

  we have

  $$\begin{array}{lll}\dfrac{\partial {\mathcal{H}}}{\partial c}&=&\dfrac{\partial u}{\partial c}+\gamma \dfrac{\partial g}{\partial c}\\ &=&(\,{f}_{0}-\gamma {f}_{0}^{2}+\gamma {f}_{0})\dfrac{\partial {\lambda }_{0}}{\partial c}-\gamma {f}_{0}\dfrac{\partial \mu }{\partial c}+\gamma (1-{f}_{0})\dfrac{\partial \nu }{\partial c}.\end{array}$$

  Now,

  $$\begin{array}{lll}{\lambda }_{0}(c)={\lambda }_{0}-\Delta {d}_{0}\cdot \dfrac{c}{c+1}\quad\Rightarrow \quad \dfrac{\partial {\lambda }_{0}}{\partial c}=-\dfrac{\Delta {d}_{0}}{{(c+1)}^{2}},\\\qquad\quad\;\; \mu (c)={\mu }_{0}+kc\quad\Rightarrow \quad \dfrac{\partial \mu }{\partial c}=k,\\\qquad\quad\;\; \nu (c)={\nu }_{0}-mc\;\;\;\Rightarrow \quad \dfrac{\partial \nu }{\partial c}=-m.\end{array}$$

  If we set

  $$\begin{array}{rcl}a(\,{f}_{0},\gamma )&:= &\Delta {d}_{0}\,{f}_{0}(\gamma {f}_{0}-1-\gamma ),\\ b(\,{f}_{0},\gamma )&:= &-\gamma k{f}_{0}+\gamma m({f}_{0}-1),\\ \end{array}$$

  we then have

  $$\frac{\partial {\mathcal{H}}}{\partial c}=\frac{a}{{\left(c+1\right)}^{2}}+b,$$

  and we obtain the second-degree polynomial equation

  $$b{c}^{2}+2bc+(a+b)=0.$$

• (6) Steps (3)–(5) are repeated until convergence.

Figure 2 shows optimal dosing strategies under linear induction of tolerance computed using our implementation of the forward-backward sweep method. These results are consistent with computations obtained using the Dymos optimal control library in Python together with the IPOPT solver, as shown in Supplementary Fig. 3.

Rapid pulsed strategies

Pulsed schedules are simple treatment strategies which alternate between giving a fixed drug dose and no drug. We identify a pulsed schedule as a three-dimensional vector (c, t_cycle, φ) where c is the dose applied, t_cycle is the duration of each treatment cycle, and φ is the proportion of drug application during each cycle. We will be particularly interested in idealized strategies where the treatment cycles are arbitrarily short. We note that if the cycle time t_cycle = dt is infinitesimal, then using (10), the infinitesimal change in f₀ is

$$\begin{array}{rcl}d{f}_{0}&=&\varphi dt\left(\left({\lambda }_{1}-{\lambda }_{0}(c)\right)\,{f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}(c)+\mu (c)+\nu (c)\right)\,{f}_{0}+\nu (c)\right)\\ &&+(1-\varphi )dt\left(\left({\lambda }_{1}-{\lambda }_{0}\right)\,{f}_{0}^{2}-\left({\lambda }_{1}-{\lambda }_{0}+{\mu }_{0}+{\nu }_{0}\right){f}_{0}+{\nu }_{0}\right).\end{array}$$

If we define the average rates over each cycle,

$$\begin{array}{rcl}{\bar{\lambda }}_{0}(c,\varphi )&:= &\varphi {\lambda }_{0}(c)+(1-\varphi ){\lambda }_{0}={\lambda }_{0}+\varphi ({\lambda }_{0}(c)-{\lambda }_{0}),\\ \bar{\mu }(c,\varphi )&:= &\varphi \mu (c)+(1-\varphi ){\mu }_{0}={\mu }_{0}+\varphi (\mu (c)-{\mu }_{0}),\\ \bar{\nu }(c,\varphi )&:= &\varphi \nu (c)+(1-\varphi ){\nu }_{0}={\nu }_{0}+\varphi (\nu (c)-{\nu }_{0}),\end{array}$$

(22)

then f₀(t) obeys the differential equation

$$\begin{array}{l}\dfrac{d{f}_{0}}{dt}=\left({\lambda }_{1}-{\bar{\lambda }}_{0}(c,\varphi )\right){f}_{0}^{2}-\left({\lambda }_{1}-{\bar{\lambda }}_{0}(c,\varphi )+\bar{\mu }(c,\varphi )+\bar{\nu }(c,\varphi )\right)\\\qquad{f}_{0}+\bar{\nu }(c,\varphi ).\end{array}$$

(23)

Similarly, the instantaneous growth rate of the population is given by

$$\bar{u}(c,\varphi ,{f}_{0})=({\bar{\lambda }}_{0}(c,\varphi )-{\lambda }_{1})\,{f}_{0}+{\lambda }_{1}.$$

Thus, under an arbitrarily fast pulsed schedule, the associated model dynamics can be approximated by a constant-dose model with parameters ${\bar{\lambda }}_{0}(c,\varphi )$, λ₁, $\bar{\mu }(c,\varphi )$ and $\bar{\nu }(c,\varphi )$. By the section on behavior under a constant dose above, in the long run, the population grows at exponential rate

$$\begin{array}{lll}\sigma (c,\varphi )\;:=\,\dfrac{1}{2}\bigg(({\bar{\lambda }}_{0}(c,\varphi )-\bar{\mu }(c,\varphi ))+({\lambda }_{1}-\bar{\nu }(c,\varphi ))\\\qquad\qquad\quad+\left.\sqrt{{\left(({\bar{\lambda }}_{0}(c,\varphi )-\bar{\mu }(c,\varphi ))-({\lambda }_{1}-\bar{\nu }(c,\varphi ))\right)}^{2}+4\bar{\mu }(c,\varphi )\bar{\nu }(c,\varphi )}\right).\end{array}$$

(24)

In addition, the population eventually reaches an equilibrium composition, where the proportion of sensitive cells becomes

$${\bar{f}}_{0}(c,\varphi ):= \frac{\bar{\nu }(c,\varphi )}{\sigma (c,\varphi )-{\bar{\lambda }}_{0}(c,\varphi )+\bar{\mu }(c,\varphi )+\bar{\nu }(c,\varphi )}.$$

(25)

Baseline parameters

Our computational results are shown using a parametrization of the model inspired by experimental investigations of drug tolerance in cancer and bacteria. The parameter values are shown in Table 1. First, we assume that in the absence of drug, the proliferation rate of sensitive cells is λ₀ = 0.04, and that the maximal drug effect on the death rate is Δd₀ = 0.08. Our motivating example is PC9 cells treated with a large dose of erlotinib, for which the proliferation rate of sensitive cells in the absence of drug has been estimated as λ₀ = 0.04 per h and the drug effect as Δd₀ = 0.08 per h¹⁴. We note that for a cohort of patients with metastatic melanoma treated with vemurafenib, the typical value of λ₀ was 0.01 per day and the typical drug effect was Δd₀ = 0.04 per day. Thus, the time unit for the baseline regime can be considered to be hours in the in vitro setting and days in the clinical setting.

Table 1 Baseline parameter values used in the study

Second, we assume that in the absence of drug, λ₁ ≪ λ₀, meaning that the drug-tolerant cells proliferate much slower than the drug-sensitive cells, and that they only make up a small proportion of the tumor at the start of the treatment. This is consistent with experimental evidence showing that drug-tolerant cells are generally slow-cycling^{7,10,11,15,57,58}. Third, we assume that μ₀ ≪ λ₀, meaning that transitions from sensitivity to tolerance are rare compared to cell divisions. This is consistent with evidence that epigenetic modifications are commonly retained for 10 − 10⁵ cell divisions^59,60,61. Fourth, we assume that ν₀ is significantly larger than μ₀, meaning that the tolerant state is lost faster than it is adopted. This is consistent with experimental evidence both from cancer^62,63 and bacteria⁶⁴.

In Figs. 2 and 3, we assume that k = m = 0.0004 and that the maximum allowable dose is ${c}_{\max }=10$ as a proportion of the ${{\rm{EC}}}_{50}^{d}$ dose. In Fig. 4, we assume that Δμ = 0.004, Δν = 0.003 and ${c}_{\max }=10$. Unless otherwise noted, we assume that the proportion of sensitive cells at the start of treatment is the equilibrium proportion ${\bar{f}}_{0}(0)$ in the absence of drug.

We note that with this parametrization, the net growth rate of tolerant cells taking phenotypic switching into account is λ₁ − ν₀ = −0.003 < 0. Thus, in the presence of drug, the tolerant population decays over time, but much slower than the sensitive cells.

Implementation of the Russo et al. model

Model and parametrization

In Russo et al.¹⁷, the authors use their experimental data to parametrize a mathematical model involving two cell types, sensitive and tolerant. The drug effect on the sensitive cell death rate d₀ is modeled using a function of the form

$${d}_{0}(c)={d}_{0}+({d}_{\max }-{d}_{0})(1-\exp (-ac))={d}_{0}+\Delta {d}_{0}(1-\exp (-ac)),$$

where c is the drug dose measured in M. Under this model, the dose which has half the maximal effect on the sensitive cell death rate is given by ${{\rm{EC}}}_{50}^{d}=\log (2)/a$. If we measure the drug dose as a proportion of the ${{\rm{EC}}}_{50}^{d}$ dose, we can rewrite the above function as

$${d}_{0}(c)={d}_{0}+\Delta {d}_{0}(1-\exp (-c\log (2))).$$

(26)

We note that this version of d₀(c) is concave in c, same as the functional form we assume in the section on the drug effect on cell proliferation above.

For the WiDr cell line, Russo et al. infer that the transition rate μ(c) from sensitivity to tolerance is given by the linear function

$$\mu (c)=k^{\prime} c,$$

where c is the drug dose measured in M. If we again measure the drug dose as a proportion of the ${{\rm{EC}}}_{50}^{d}$ dose and define $k:= k^{\prime} {{\rm{EC}}}_{50}$, we can rewrite this function as

$$\mu (c)=kc.$$

With this modeling setup, the authors in ref. ¹⁷ use their experimental data to derive the parameter estimates λ₀ = b₀ − d₀ = 0.048, a = 2.91495 × 10⁶, Δd₀ = 1.095, λ₁ − ν₀ = −0.073 and $k^{\prime} =136935$ for the WiDr B7 clone. For these estimates, the ${{\rm{EC}}}_{50}^{d}$ dose is given by

$${{\rm{EC}}}_{50}^{d}=\frac{\log (2)}{2.91495\cdot 1{0}^{6}}=2.3779\cdot 1{0}^{-7}\ M,$$

and the rescaled slope k is given by

$$k=136935\cdot {{\rm{EC}}}_{50}^{d}=0.03256.$$

Implementation of forward-backward sweep method

To solve the optimal control problem (16) using the drug effect function in (26), we can apply the forward–backward sweep method as already laid out with one modification. In the fifth step, we have to solve $\partial {\mathcal{H}}/\partial c=0$. We again have that

$$\frac{\partial {\mathcal{H}}}{\partial c}=(\,{f}_{0}-\gamma {f}_{0}^{2}+\gamma {f}_{0})\frac{\partial {\lambda }_{0}}{\partial c}-\gamma {f}_{0}\frac{\partial \mu }{\partial c}+\gamma (1-{f}_{0})\frac{\partial \nu }{\partial c},$$

but we must modify our calculation of ∂λ₀/∂c. In this case, we have

$${\lambda }_{0}(c)={\lambda }_{0}-\Delta {d}_{0}(1-\exp (-c\log (2))),$$

which implies that

$$\frac{\partial {\lambda }_{0}}{\partial c}=-\log (2)(\Delta {d}_{0})\exp (-c\log (2)),$$

and we obtain the equation

$$-\log (2)(\Delta {d}_{0})\exp (-c\log (2))(\,{f}_{0}-\gamma {f}_{0}^{2}+\gamma {f}_{0})-\gamma {f}_{0}k=0.$$

If we now set

$$\begin{array}{lll}a(\,{f}_{0},\gamma )\;:=\,\log (2)(\Delta {d}_{0}){f}_{0}(\gamma {f}_{0}-1-\gamma ),\\ b(\,{f}_{0},\gamma )\;\,:=\,-\gamma {f}_{0}k,\end{array}$$

we obtain

$$\exp (-c\log (2))=-\frac{b}{a},$$

which yields $c=-\log (-b/a)/\log (2)$. This is the solution we use in the fifth step.

Natural transitions between phenotypes

In Russo et al.¹⁷, the authors assume no transitions from sensitivity to tolerance in the absence of drug, μ₀ = 0. For the WiDr cell line (clone B7), their point estimate for the proportion of sensitive cells at the start of treatment is 0.992, which would indicate transitions between phenotypes before treatment. However, the proportion is indistinguishable from zero when statistical uncertainty is taken into account. The authors point out that transitions from tolerance to sensitivity may occur even in the presence of drug, but that these transitions would effectively contribute to the death rate of tolerant cells. Overall, it is uncertain to what extent cells transition between sensitivity and tolerance in the absence of drug.

To be consistent with our modeling framework, we assume transitions between types do occur in the absence of drug, so that they lead to an equilibrium proportion ${\bar{f}}_{0}(0)=0.992$ of sensitive cells in the absence of drug. This leads us to take μ₀ = 0.001, λ₁ = 0 and ν₀ = 0.073, which is still consistent with Russo et al.’s finding that drug tolerance is primarily induced by the drug and that λ₁ − ν₀ = −0.073. A summary of the parameter values used for our analysis of the Russo et al. model is given in Table 2.

Table 2 Parameter values used in our analysis of the Russo et al. model