Delaying cancer progression by integrating toxicity constraints in a model of adaptive therapy

Abstract

Cancer therapies often fail when intolerable toxicity or drug-resistant cancer cells undermine otherwise effective treatment strategies. Over the past decade, adaptive therapy has emerged as a promising approach to postpone emergence of resistance by altering dose timing based on tumor burden thresholds. Despite encouraging results, these protocols often overlook the crucial role of toxicity-induced treatment breaks, which may permit tumor regrowth. Herein, we explore the following question: would incorporating toxicity feedback improve or hinder the efficacy of adaptive therapy? To address this question, we propose a mathematical framework for incorporating toxic feedback into treatment design. We find that the degree of competition between sensitive and resistant populations, along with the growth rate of resistant cells, critically modulates the impact of toxicity feedback on time to progression. Further, our conceptual model identifies circumstances where strategic treatment breaks, which may be based on either tumor size or toxicity, can mitigate overtreatment and extend time to progression, both at the baseline parameterization and across a heterogeneous virtual population. Taken together, these findings highlight the importance of integrating toxicity considerations into the design of adaptive therapy.

Introduction

Systemic anti-cancer therapies, including chemotherapy, targeted therapies, immunotherapies, and hormone treatments, are essential components of modern oncology. In patients with metastatic disease, where cure is often elusive, the primary goals are to control tumor growth, extend survival, minimize treatment-related toxicities, and alleviate cancer-associated symptoms¹. Typically, a maximum tolerated dose (MTD) protocol is used to achieve these goals. The goal of MTD is to eliminate as many cancer cells as possible by administering the drug at the highest dose deemed safe until toxicity limitations are reached.

Despite significant advances in the precision and effectiveness of systemic anti-cancer therapies, two persistent challenges remain. The first is the management of treatment-induced toxicity^2,3,4. Toxicities associated with MTD treatment include fatigue, gastrointestinal side effects, and hematological complications^5,6,7,8,9. These toxicities adversely affect patient quality of life and often necessitate dose reductions, treatment interruptions, or even discontinuation, all of which ultimately undermine therapeutic effectiveness.

The second persistent challenge of MTD therapy is the emergence of therapeutic resistance. In clinical practice, a tumor is considered sensitive if it shrinks or remains stable with treatment, and resistant if it continues to grow. However, because tumors are inherently heterogeneous, they typically contain subpopulations of cells with varying treatment sensitivities¹⁰. Under the sustained treatment typical of MTD therapy, the elimination of sensitive cells can create an environment in which resistant clones proliferate freely. This phenomenon, known as competitive release, accelerates resistance-driven progression and diminishes treatment efficacy over time^11,12.

Adaptive therapy (AT) has emerged as a promising alternative to traditional MTD approaches. Rather than aiming for complete tumor eradication, AT strategically modulates dosing schedules to preserve a population of sensitive cells that can suppress resistant clones through intra-tumor competition^13,14. Its implementation is governed by clinical decision rules based on measurable indicators, such as tumor size or circulating biomarkers. For example, a dose-skipping regimen administers a high dose until a predetermined tumor response (e.g., a 50% reduction in size) is achieved and then pauses treatment until the tumor regrows to a defined threshold, often its initial size. Alternatively, a dose-modulation strategy adjusts treatment doses incrementally, increasing or decreasing them on the basis of the tumor’s response over time¹⁵.

The effectiveness of AT can be enhanced if resistance comes with a fitness cost. This occurs when the mechanisms conferring resistance, such as the upregulation of energy-intensive drug efflux pumps, reduce a cell’s overall fitness in the absence of therapy^16,17,18. Indeed, adaptive therapies have already been tested experimentally^19,20,21 and are currently being applied across multiple clinical trials (NCT02415621; NCT03511196; NCT03543969; NCT03630120).

A crucial, yet underexplored, aspect of AT is the impact of treatment-induced toxicity on dosing decisions. Toxicity constraints can limit treatment frequency or intensity, potentially allowing both sensitive and resistant cell populations to proliferate during treatment interruptions. This could disrupt the competitive dynamics that are essential to AT’s success. Equally, integrating toxicity feedback into dosing strategies could improve patient tolerability and enable prolonged treatment, ultimately enhancing outcomes.

Attempts to model toxicity in the literature mainly focus on white blood cell counts and neutropenia induced by chemotherapy^{22,23,24,25,26}. In ref. ²⁷, the authors model chemo-induced toxicity effects as a loss of muscle mass, and in ref. ²⁸, the authors model toxicity effects via both weight loss and a comprehensive index that takes into account 57 side effects. Toxicity has also been included as an additional constraint in the design of optimal control dosing strategies^29,30.

Approaches to model toxicity include an assumption that the effect is present and constant during the whole treatment phase^22,23, that it is proportional to drug concentration^25,29,30, or that it is proportional to drug and tumor burden²⁸. In ref. ²⁴, the toxic effect of the drug directly reduces the neutrophil proliferation rate via an inverse polynomial form of the drug concentration. In ref. ²², a stochastic model is introduced wherein toxicity is modeled as an increase in the death rate of white blood cells. Through a virtual clinical trial, the authors demonstrate that managing toxicity throughout treatment by modulating or pausing doses does not compromise overall treatment outcomes. In ref. ³¹, the authors use a modified logistic competition model with sensitive and resistant cells to show that in numerical simulations, continuously-dosed AT outperforms discretely-dosed AT in terms of both time to disease progression and controlling toxicity. Their results assume toxicity is a nonlinear function of drug dose satisfying specific conditions (such as being strictly increasing and concave up).

Previously considered control problems³⁰ introduce toxicity as a constraint, but do not monitor the emergence of drug resistance or the strategy of AT. Dose interruptions in a model of resistance is presented in ref. ²², but in that paper, toxicity is considered as a constant effect that reduces white blood cell growth, and is not dependent on drug concentration or accumulated exposure. In ref. ³¹, where the authors consider continuous or intermittent adaptive therapies to manage emergence of resistance, toxicity is modeled as a side effect that can be used to determine when too much drug has been administered but is not included in dosing decision making.

In this work, we explore the following critical question: would incorporating toxicity feedback improve or hinder the efficacy of AT? To address this question, we propose a novel mathematical framework for including toxicity both in the decision-making process of daily therapy, and in the formulation of adaptive regimes based on maintaining the tumor size within a pre-specified range. Our simulations, designed to mimic a theoretical murine study with an unspecified chemotherapeutic agent, compare the impact of toxicity constraints on time to progression (TTP) for daily treatment versus AT schedules. Our results demonstrate that the degree of competition between sensitive and resistant populations, along with the growth rate of resistant cells, critically modulates how TTP is impacted by the incorporation of toxicity feedback. Notably, our conceptual model identifies circumstances where strategic treatment breaks, which may be based on either tumor size or toxicity, can mitigate overtreatment and extend TTP, both at the baseline parameterization and across a virtual population. We conclude with a discussion of the model’s limitations and propose future directions aimed at optimizing cancer treatment strategies to simultaneously address resistance and toxicity, thereby improving patient quality of life.

Results

To investigate the relationship between AT and toxicity, we analyze the dynamics captured by a mathematical model that integrates drug pharmacokinetics and toxicity dynamics into an existing framework¹⁶ of competitive interactions between treatment-sensitive ($S(t)$) and treatment-resistant ($R(t)$) cancer cells. This conceptual model, described in detail in the Methods, assumes that both subpopulations grow logistically to a shared carrying capacity and that only sensitive cells are killed by the drug. It further assumes that drug pharmacokinetics are described by a one-compartment model and that toxicity increases at a rate proportional to drug concentration and resolves at a linear rate.

In this section, we evaluate the impact of the four proposed protocols (Fig. 1) on treatment dynamics and time-to-progression (TTP): (i) daily protocol, (ii) adaptive protocol, (iii) daily protocol with toxicity feedback, and (iv) adaptive protocol with toxicity feedback. More details on the implementation of these protocols are provided in the Methods section.

Fig. 1: Summary of the four proposed treatment protocols.

A daily protocol, B adaptive protocol with tumor size-driven treatment cycles, C daily protocol with toxicity-driven treatment cycles, and D adaptive protocol with both tumor size- and toxicity-driven treatment cycles.

In all simulations, parameters are fixed at their baseline values defined in Table 1 of the Methods, except for those being explicitly varied in a parameter sweep, with the same normalized dose administered across all treatment protocols. We monitor tumor composition (fractions of sensitive and resistant cells), drug concentration, and accumulated toxicity. Protocol effectiveness is assessed based on TTP—the longer a tumor takes to fail treatment, the more effective the protocol is deemed to be.

Table 1 Baseline parameter values used in Eqs. (1)–(4)

Addition of Toxicity Constraints Can Extend TTP

In Fig. 2, we show representative model dynamics from a simulated experiment comparing the four treatment protocols using parameter values from Table 1. Each row corresponds to a protocol, with tumor cell time-courses plotted in the left panels, and drug concentration and toxicity time-courses in the right panels. Periods of drug administration are shaded in gray, and simulations run until the disease progresses. For each protocol, representative examples of model dynamics at alternative parameterizations are shown in Supplementary Fig. 1.

Fig. 2: The four treatment protocols and resulting model outputs using parameter values from Table 1.

The left column shows the time course of the total tumor size, along with the sensitive and resistant subpopulations. The right column shows the dynamics of drug concentration and accumulated toxicity. Compared protocols are daily (A,B), adaptive (C,D), daily with toxicity feedback (E,F), and adaptive with toxicity feedback (G,H). Time periods shaded gray represent when the treatment is on.

In the daily protocol (Fig. 2, top row), the drug rapidly depletes sensitive cells (blue curve), resulting in a large initial reduction in total tumor burden. However, this depletion removes intra-tumor competition, allowing resistant cells (red curve) to proliferate unchecked, leading to swift treatment failure. High and sustained drug exposure results in persistently elevated toxicity throughout treatment. This protocol yields the shortest TTP (19.2 days), creating a “lose-lose” scenario where resistance emerges quickly, and toxicity remains critically high.

In the adaptive protocol, tumor size-based feedback modulates treatment cycles, alternating between drug administration and off-treatment periods (Fig. 2, second row). This strategy allows sensitive cells to recover during treatment pauses, enabling them to compete with and temporarily suppress resistant cells, thereby delaying treatment failure. Periodic drug holidays also reduce toxicity compared to the daily protocol, though overall toxicity continues to increase with time. AT extends TTP to 44 days, more than doubling the duration achieved under the daily protocol. However, as resistant cells gradually escape suppression by the sensitive cells, the adaptive protocol ultimately loses effectiveness.

An additional observation in the adaptive protocol (Fig. 2C) is that the tumor volume sometimes overshoots the Rx_on threshold by a large margin before therapy resumes, while in other cycles the overshoot is much smaller. This irregularity is not seen in the protocols that include toxicity feedback. The explanation lies in the interaction between continuous tumor growth and the discrete, once-daily treatment decision schedule. In the adaptive protocol, the tumor may cross the Rx_on threshold at any time between daily checks; if this occurs early in the off-treatment interval, the tumor has nearly a full day to continue growing before therapy resumes, leading to a larger overshoot. If the threshold is crossed later in the interval, the overshoot is smaller. In contrast, when toxicity feedback is incorporated, therapy cannot restart immediately after tumor regrowth, and one must wait for toxicity to decline below Tox_on. This constraint effectively prolongs and regularizes the off-treatment intervals. Similar effects of overshooting thresholds have been observed in recent work by Gallagher et al.³², where AT protocols with discrete monitoring intervals often allow the tumor to exceed the upper size threshold before treatment is resumed, due to the inherent delay between checks.

Incorporating toxicity-based dose adjustments into the daily protocol introduces periodic drug holidays (Fig. 2, third row). These treatment interruptions, designed to allow recovery from drug toxicity, also enable partial regrowth of sensitive cells, creating an unintended adaptative therapy protocol. Interestingly, for our baseline parameter values, this approach achieves a longer TTP (59.8 days) than even the adaptive protocol. This is because the toxicity feedback effectively mimics an adaptive strategy with higher Rx_on thresholds; that is, treatment resumes only after the tumor has grown beyond a threshold that would have triggered re-administration in the adaptive protocol.

The adaptive protocol with toxicity modulation integrates both tumor size-based and toxicity-based treatment decisions (Fig. 2, bottom row). This doubly adaptive approach results in sustained on/off treatment cycles. Like the daily protocol with toxicity feedback, tumor sizes overshoot the Rx_on threshold because treatment cannot resume until toxicity has declined sufficiently. This has the dual effect of reducing drug exposure, and consequently, toxicity, while preserving the sensitive cell population during early cycles. As a result, this protocol achieves the longest TTP (79.1 days).

Sensitivity analysis

Next, we perform a global sensitivity analysis using eFAST to quantify the impact of parameter variations on TTP across the four treatment protocols. The results are summarized in Fig. 3. It should be noted that eFAST assigns a minimal sensitivity value to all parameters, including dummy parameters, making the p-values (indicated by asterisks in each plot) crucial for interpreting the significance of each sensitivity index.

Fig. 3: eFAST sensitivity indices as quantified by TTP.

Result for A daily protocol, B daily protocol with toxicity, C adaptive protocol, and D adaptive protocol with toxicity. Asterisks indicate statistical significance: * p < 0.05, ** p < 0.01, *** p < 0.001. Parameter ranges used are indicated in the graph. Note that ${\alpha }_{R}=\epsilon {\alpha }_{S}$.

The sensitivity indices plotted in Fig. 3 indicate that no single parameter strongly influences TTP in all four protocols. In the adaptive protocols (Fig. 3C, D), the competition parameter $\beta$ most significantly influences TTP, as indicated by having the largest first order (blue bars), and higher order (orange bars), sensitivity indices. In the daily protocols (Fig. 3A, B), the parameter with the most influence TTP is the relative growth rate of resistant cells relative to sensitive cells ($\epsilon ,{\alpha }_{R}=\epsilon {\alpha }_{S}$). Counterintuitively, the rate of toxicity recovery, $\gamma$, is either the least or second least sensitive parameter in any of the protocols. We explore this further in ‘Effect of Parameters on TTP’ subsection.

Effect of Parameters on TTP

Parameters $\beta$ and $\epsilon$ (or ${\alpha }_{R}$ as ${\alpha }_{R}=\epsilon {\alpha }_{S}$) were identified as significant for most protocols in Fig. 3, so we start our analysis with them. We first investigate the effect on TTP of varying $\beta$ and ${\alpha }_{R}$ over the ranges defined in the global sensitivity analysis, with all other parameters fixed at their baseline values from Table 1. For each pair of values $(\beta ,{\alpha }_{R})$ we compute the TTP for each protocol (Supplementary Fig. 2). The results of these four parameter sweeps are summarized in Fig. 4. In particular, Fig. 4A indicates which protocol(s) result in the longest TTP, and Fig. 4B indicates the actual time to progression for the protocol(s) that maximized TTP. Recall that the simulation is run for 100 days. We assign a value of 150 to any simulation that does not reach treatment failure within this simulation time.

Fig. 4: Parameter sweeps over β–α_R space.

A Protocol that maximizes time to progression for the corresponding pair of parameter values. B Maximum time to progression for the optimal protocol(s) identified in (A). A value of 150 indicates that the predicted tumor did not progress within the simulation time of 100 days and thus, that the treatment protocol did not (yet) fail. The baseline parameterization is indicated with a red star. Parameters that are not varied are fixed at their baseline value in Table 1 with ${\alpha }_{R}$ varied from 0.2 to 1 by varying $\epsilon$.

As shown in Fig. 4A, one of the two adaptive protocols is overwhelmingly likely to maximize TTP when the resistant cells have a competitive advantage over the sensitive cells ($\beta < 1$). The only exception to this is when resistant cells grow at nearly the same rate as the sensitive cells (${\alpha }_{R}$ near 1). In this scenario, the competitive advantage of the resistant cells is so extreme that none of the protocols can contain the tumor, and any stopping of treatment decreases TTP. Figure 4B shows that as the competitive advantage of the resistant cells increases (smaller $\beta$, larger ${\alpha }_{R}$), the optimal protocol becomes less effective. As resistant cells lose their direct competitive advantage over sensitive cells (in particular for $1\le \beta \le 2$), the incorporation of toxicity feedback into the adaptive protocol yields the optimal result, though treatment still fails within the simulation period. Finally, when sensitive cells have a very strong competitive advantage over resistant cells ($\beta \ge 2.75$) or a sufficiently strong competitive advantage ($2.25\le \beta \le 2.5$) coupled with significantly weaker growth of resistant cells relative to sensitive cells (small $\epsilon$), TTP extends beyond the simulation window. As we move towards the parameter regime where the tumor does not progress (higher $\beta$) from our baseline parameterization (red star), the adaptive protocol with toxicity modulation is the only protocol that appears as optimal (protocol 4 in Fig. 4A). As we move further to the right, we interestingly find that it is the daily protocol with toxicity modulation that can also extend TTP beyond the simulation period (protocol 3 in Fig. 4A). Finally, increasing $\beta$ further, moving to the extreme end of parameter space for which sensitive cells have a competitive advantage over resistant cells, the adaptive protocol joins the two toxicity protocols as all three prevent progression within the simulation period.

Rates of drug-induced death and of toxicity recovery affect the nature of treatment failure

Next, we want to assess whether the cause of treatment failure is due to resistant or sensitive cells. Surprisingly, simulations revealed that while ${\alpha }_{R}$ (varied by changing $\epsilon$) strongly influences time to failure, it does not play an important role in determining the cause of failure (data not shown). This is because treatment failure by resistant cells or sensitive cells depends primarily on whether toxicity constrains the ability to control sensitive cells which is relatively independent of the intrinsic growth rate of the resistance cells.

To explore the underlying cause of progression, we performed a parameter sweep in $\left(\beta ,\delta \right)$-space, where $\delta$ is the drug-induced death rate and $\beta$ is the competitive effect of sensitive cells on resistant cells. For each protocol and each $(\beta ,\delta )$ pair, keeping all other parameters fixed at their baseline values, we assess tumor composition (fraction of resistant cells) at the time of treatment failure. In the protocols without toxicity (Fig. 5A, B) treatment failure results from the eventual dominance of resistant cells, with sensitive cells nearly eradicated.

Fig. 5: Fraction of resistant cells at TTP for different combinations of β and δ tested over the four protocols.

Results for A daily, B adaptive, C daily with toxicity, and D adaptive with toxicity feedback. All other parameters are fixed at their baseline values reported in Table 1. Black regions indicate that treatment did not fail within the simulation time of 100 days. As expected, progression when it occurs is always from resistant cells when there are no toxicity constraints. With toxicity constraints, as the rate of drug-induced cell mortality ($\delta$) declines, the cause of progression switches from resistant to sensitive cells.

With toxicity, drug doses may be constrained to the point where sensitive cells drive progression regardless of treatment strategy (daily versus adaptive in Fig. 5C, D, respectively). When drug efficacy is low (small $\delta$ values), toxicity prevents sufficient drug dosing to control the sensitive cells and they drive progression. The switch from resistant to sensitive cells causing progression is relatively abrupt, occurring close to $\delta =1$. Despite the importance of $\beta$ in determining TTP (larger values prolong TTP), the role of $\beta$ in the cause of treatment failure is minimal. However, the transition from resistant to sensitive cells causing progression as $\delta$ declines becomes less abrupt at small values of $\beta$.

The global sensitivity analysis in Fig. 3 identified the toxicity recovery rate $\gamma$ as either the least sensitive, or second least sensitive, parameter in all four protocols. This is surprising, as one would think that the rate of toxicity recovery would affect the outcomes of protocols with toxicity feedback. If the rate of toxicity recovery does not significantly influence TTP, perhaps it affects the cause of treatment failure? We explore this question next.

Figure 6 assesses the effect of $\gamma$ in two different parameter regimes identified from Fig. 5C, D: when sensitive cells drive progression ($\delta =2/3$) and when resistant cells drive progression ($\delta =4/3$). In both parameter regimes, we observe a bifurcation in the reason for treatment failure. When $\gamma$ is small toxicity accumulates, forcing treatment breaks that allow sensitive cells to regrow and outcompete resistant cells, ultimately driving treatment failure. Conversely, when $\gamma$ is large, toxicity resolves quickly, permitting more frequent treatment cycles that, in turn, eliminate sensitive cells leaving the resistant cells to drive treatment failure. The bifurcation in behavior occurs at the same value of $\gamma$ for each value of $\delta$ considered (compare Fig. 6A, B with $\delta =2/3$, then compare Fig. 6C, D with $\delta =4/3$).

Fig. 6: Effect of recovery from toxicity, γ, on the nature of treatment failure.

As $\gamma$ increases, the reason for treatment failure switches from sensitive to resistant cells. A Daily with toxicity and B adaptive with toxicity protocols when drug effectiveness, $\delta$, is small ($\delta =2/3$). C Daily with toxicity and D adaptive with toxicity protocols when $\delta$ is large ($\delta =4/3$). The remaining parameters are fixed at their baseline value in Table 1.

Interestingly, there is a range of intermediate $\gamma$ values in the $\delta =2/3$ case, where treatment does not fail within the simulation time frame. These cases result in prolonged on-off cycling of therapy. With this moderate drug efficacy, toxicity can be controlled and the resulting feedback-modified treatments work well. However, at higher values of treatment efficacy ($\delta$), this ‘Goldilocks’ window vanishes, and treatment invariably fails during the simulation period. Prolonged cycling of treatment is no longer a possibility as resistant cells will lead to treatment failure. These results demonstrate that even though $\gamma$ has a minimal effect on TTP (sensitivity analysis in Fig. 3), it can play a crucial role in determining the cause of treatment failure.

Assessing therapy outcome robustness to protocol thresholds

Next, we assess whether treatment outcomes can be improved by adjusting the therapy on/off thresholds in adaptive and toxicity-tracking protocols. We previously showed that at baseline tumor size and toxicity thresholds for starting or stopping therapy, TTP varies significantly with $\beta$ and ${\alpha }_{R}$. Here, we fix all model parameters to the baseline values from Table 1 and instead vary the tumor size thresholds Rx_on and Rx_off in the adaptive protocols, or the toxicity Tox_on and Tox_off in the protocols with toxicity feedback. We report predicted TTP for each of the following cases: the effect of varying Tox_on and Tox_off in daily and adaptive protocols with toxicity feedback (Fig. 7A, B, respectively) and the effect of varying Rx_on and Rx_off in adaptive protocols without or with toxicity feedback (Fig. 7C, D, respectively).

Fig. 7: TTP as a function of protocol thresholds at baseline parameter values.

Top row: toxicity thresholds varied in the daily with toxicity (A) and adaptive with toxicity (B) protocols. Bottom row: tumor size thresholds varied in the adaptive (C) and adaptive with toxicity (D) protocols. Parameter sets for which the constraint Rx_on > Rx_off is not satisfied are blacked out. The results of the same protocol sweep for smaller and larger values of $\beta$ than the baseline value of $\beta =2.4$ used here are shown in Supplementary Fig. 3.

Figure 7A, B show TTP as a function of Tox_on (the toxicity level at which treatment may resume, horizontal axis) and Tox_off (the toxicity level at which therapy is paused, vertical axis). The constraint Tox_on < Tox_off must be satisfied; protocols violating this condition are blacked out in the bottom right corners. When Tox_off is too high, that is, when we are allowing a high level of toxicity in the simulation, toxicity-induced treatment pauses are infrequent. As a result, TTP in toxicity-feedback protocols is similar to that of protocols without toxicity feedback in both daily (TTP of 19.2 days) and adaptive (TTP of 44 days) dosing. Low values of Tox_on, meaning treatment can only resume after toxicity has mostly resolved, result in sharp declines in TTP. In these cases, if toxicity triggers a treatment pause (as seen with moderate Tox_off values in the daily protocol, and all values in the adaptive protocol), the toxicity level never falls low enough for the treatment to restart. The result is that cancer cells grow unchecked and TTP is very short. Conversely, high values of Tox_on allow for treatment to be resumed sooner, minimizing the effect of toxic feedback on the protocols, and resulting in similar TTPs to that of protocols without toxicity feedback.

Notably, the longest TTP for daily and adaptive protocols with toxicity feedback occurs at intermediate toxicity thresholds that are fairly close together, forcing toxicity to remain relatively constant and moderate. Within this sweet spot in protocol space, the model predicts that the tumor will not progress during the 100-day simulation window. These protocols with an optimal TTP occur at the boundary of the blacked-out region (Fig. 7A, B). Interestingly, this tumor control behavior occurs over a larger range of protocol thresholds for the adaptive with toxicity protocol, as compared to the daily with toxicity protocol.

Figure 7C, D show TTP as a function of Rx_on (the tumor size at which treatment resumes, horizontal axis) and Rx_off (the tumor size at which treatment pauses, vertical axis). The constraint Rx_on > Rx_off must be satisfied; parameter sets violating this condition are blacked out in the top left corners. Looking at the rightmost columns in Fig. 7C, D, when Rx_on is too high, the tumor is allowed to grow to a sufficiently large size before a new treatment cycle begins. Under both adaptive protocols, TTP is noticeably lower than in the corresponding daily protocols (reported in the title of each subfigure), regardless of Rx_off. When Rx_on is small, both adaptive protocols (with or without toxicity feedback) are comparable to, or modestly outperform, the corresponding daily protocol.

As we saw when varying the toxicity thresholds, there is a sweet spot in the adaptive protocol space for which the tumor is controlled over the 100-day window. This optimal window is found at high Rx_off and intermediate Rx_on values (indicated by the boxes labeled ‘150’ in Fig. 7C). This region suggests that the best outcome is obtained by pausing treatment before the tumor shrinks too much, thereby allowing the sensitive cell population to recover and suppress resistant cells through competition, and by restarting treatment before the tumor becomes too large.

Interestingly, we do not observe the same sweet spot in protocol space for the AT with toxicity feedback (Fig. 7D). At high Rx_off and intermediate Rx_on values, TTP is smaller than what we observe with the baseline threshold values (59.8 days). Instead, an optimal treatment window emerges at intermediate Rx_off values, where tumor burden is maintained between 36.6% and 56.6% of its initial size. Although adjusting size-based thresholds can improve TTP, none of the tested protocols produce a durable response – defined here as TTP exceeding the 100-day simulation period – once toxicity feedback is included in the adaptive protocol. While we did not identify tumor size thresholds for which the adaptive protocol with toxicity can control the tumor in Fig. 7D, Fig. 7B shows that toxicity feedback can produce the desired outcome. Both Fig. 7B, D are looking at two-dimensional slices of a four-dimensional protocol space, so it is important not to interpret this as concluding that the adaptive protocol with toxicity feedback is inferior to the others, as these are snapshots of a larger space.

The threshold sweeps in Fig. 7 also highlight the personalization level required to optimize the toxicity feedback protocol. We see that, depending on the chosen toxicity thresholds, the efficacy of a toxicity feedback protocol can improve or worsen relative to the daily protocol’s TTP of 19.2 days. As an example, consider Fig. 7A and fix the maximum tolerated toxicity at Tox_off = 1.67. Moving the treatment restart threshold Tox_on from 0.92 to 1.08 results in a jump in TTP from 12.7 days (worse than the daily protocol) to over 100 days (much better than the daily protocol). However, a further increase of Tox_on reduces the predicted TTP, revealing the existence of a toxicity threshold “sweet spot” for this fixed parametrization. Such “sweet spots” exist for all the treatment cycling protocols, and the landscapes will depend on the model parameters.

Optimizing threshold “Sweet Spots” across a virtual population

Finally, to systematically determine whether adaptive and toxicity-based thresholds can be optimized for maximal TTP across various parameter values, we conduct a virtual patient analysis. In this approach, each “virtual patient” is assigned a unique set of values for five parameters, ${\alpha }_{S}$, $\epsilon$, $\beta$, $\delta$, and $\gamma$, drawn from independent lognormal distributions (Supplementary Fig. 4). The means of these distributions match the corresponding baseline values in Table 1, and the standard deviations are chosen to approximate the parameter ranges used in the sensitivity analysis of Fig. 3. We then repeat the protocol threshold analysis from Fig. 7 for each virtual patient and calculate the mean TTP across the entire virtual population, for every pair of threshold values. As before, VPs that do not reach treatment failure in 100 days are assigned a TTP of 150. The results for 100 simulated VPs are reported in Fig. 8.

Fig. 8: Mean TTP for 100 VPs.

Top row: toxicity thresholds varied in the daily with toxicity (A) and adaptive with toxicity (B) protocols. Bottom row: tumor size thresholds varied in the adaptive (C) and adaptive with toxicity (D) protocols.

Figure 8A, B show mean TTP as a function of Tox_on (the toxicity level at which treatment resumes, horizontal axis) and Tox_off (the toxicity level at which treatment pauses, vertical axis), with the tested protocol specified in the panel title, along with the corresponding mean TTP for the protocol without toxicity feedback. Again, the blacked out bottom right corners correspond to pairs of values that violate the constraint that Tox_on < Tox_off. Figure 8C, D show mean TTP as a function of Rx_on (the tumor size at which treatment resumes, horizontal axis) and Rx_off (the tumor size at which treatment pauses, vertical axis), with the tested protocol specified in the panel title, along with the corresponding mean TTP for the protocol without tumor size-based feedback. The upper left corner is blacked out as this region violates the constraint that Rx_on > Rx_off.

It is instructive to compare the heatmaps in Fig. 7 for the baseline parameterization to the heatmaps in Fig. 8, where we are looking at average behavior across a virtual population. When the toxicity thresholds are varied, we generally see that the sweet spot in protocol space is comparable between the baseline parameterization (Fig. 7A, B) and the virtual population (Fig. 8A, B). The same is true when the tumor size thresholds are varied in the adaptive protocol (Fig. 7C versus Fig. 8C). Interestingly, the analogous consistency is not observed in the adaptive protocol with toxic feedback (Fig. 7D versus Fig. 8D). At the baseline parameter values, intermediate Rx_on values coupled with large Rx_off values corresponded to a suboptimal treatment protocol. However, this same region of protocol space is optimal for the average of the virtual population. This demonstrates that the effect of tumor size thresholds in the adaptive with toxicity protocol depends, to a large extent, on the model parameters.

To demonstrate the variability across VPs, as opposed to the average behavior as in Fig. 8, we examine the distribution of patient outcomes by plotting Kaplan-Meier curves (Fig. 9). These curves illustrate the proportion of virtual patients who have not progressed over time. The Kaplan-Meier curve for each protocol uses the thresholds that were identified to optimize the mean TTP for the virtual population in Fig. 8. For instance, the adaptive protocol in Fig. 8C has the longest TTP when Rx_on = 58.33 and Rx_off = 45, so these are the thresholds set for the adaptive protocol in Fig. 9. For reference, we also include the Kaplan-Meier curve for the daily protocol with no treatment pausing (blue curve). The adaptive with toxicity protocol (purple curve) in Fig. 9 uses the optimal thresholds found by searching over the four-dimensional space defined by the two toxicity thresholds and the two tumor size thresholds.

Fig. 9: Variability across a virtual population for each protocol with optimized treatment thresholds.

Mean TTP of 100 VPs was used to determine the optimal thresholds for each protocol (except the daily protocol which does not have any thresholds).

By adding toxicity considerations to the daily protocol (yellow curve in Fig. 9) 40% of VPs experience a TTP greater than 100 days, but 16% experience a TTP shorter than the daily protocol. The adaptive protocol (red curve) significantly improves TTP for the majority of VPs compared to the daily protocol and leaves almost 50% of VPs with no disease progression by 100 days. Finally, the adaptive with toxicity protocol (purple curve), demonstrates the potential advantage of dual feedback: 91% experience no progression by 100 days.

Discussion

Cancer treatments can fail for many reasons, including the emergence of resistance or unacceptable toxicity. In practice, toxicity often necessitates dose reductions, delays, or discontinuations, undermining the durability of systemic therapy. AT has been proposed as an alternative to conventional MTD approaches, with the aim of prolonging tumor control by maintaining a population of drug-sensitive cells that suppress resistant clones. In settings where cure is not possible due to the presence of resistant subpopulations, previous mathematical models and several clinical trials suggest that AT (e.g., pausing and restarting treatment based on tumor burden) can improve outcomes by delaying progression. However, most mathematical models of AT have focused on tumor burden alone and have not explicitly incorporated toxicity into treatment decisions. Our study addresses this gap by introducing a conceptual framework in which toxicity constraints, alongside efficacy thresholds, could govern treatment administration.

We demonstrate that toxicity constraints can enhance or diminish outcomes depending on parameter values and protocol thresholds. In our analysis, we compared TTP of four protocols: daily; adaptive, where treatment is controlled based on tumor size criteria; daily with toxicity feedback, where treatment is controlled based on accumulated toxicity criteria; and adaptive with toxicity feedback, where treatment is controlled by both tumor size and toxicity criteria. We were able to identify scenarios in which each of the four protocols could optimize TTP. The optimal protocol depended on the growth rates of, and competition between, the sensitive and resistant cells, and on the defined protocol thresholds. Protocols leading to the longest TTPs typically incorporate either or both efficacy and toxicity feedback rules.

Additionally, by assessing tumor composition at TTP, we found that there are two paths to treatment failure. The tumor may progress due to significant growth of either sensitive or resistant cells, with progression due to sensitive cells occurring only in protocols with toxicity feedback. As such, drug dose timing becomes critically important: waiting too long to resume treatment (for example, because of slow toxicity recovery), may result in sensitive cell growth causing treatment failure.

Our results demonstrate that varying toxicity and efficacy thresholds can significantly affect treatment response, with relatively small changes in threshold values potentially having significant effects on TTP. We also observed that TTP is sensitive to several model parameters. For these reasons, we conducted a virtual population analysis to better understand the relationship between protocol thresholds and TTP across a heterogeneous population. In particular, we varied the protocol thresholds for both tumor size and/or toxicity (as relevant to the protocol), to determine the optimal thresholds leading to the longest average TTP for the virtual population. We found that while no protocol can prevent progression across the entire virtual population, there do exist protocols (threshold values) that can significantly extend TTP, on average, for all protocols with feedback. The longest average TTP, with the most non-progressing VPs, was predicted for the dual-feedback protocol incorporating both adaptive and toxicity constraints, as demonstrated by the simulated Kaplan-Meier survival curves in Fig. 9.

Although the proposed model is conceptual, employing a simplified representation of tumor heterogeneity (two subpopulations), drug pharmacokinetics, resistance (pre-existing only) and treatment-induced toxicity, the results are consistent with key principles of AT. Specifically, treatment pauses based on tumor size (though not toxicity) can actually maintain or even improve efficacy³³, potentially through mitigating competition between sensitive and resistant cancer cells. By pausing or reducing treatment upon reaching specific toxicity thresholds and resuming once resolved, patients may experience fewer adverse effects without compromising treatment outcomes. The importance of such an approach is further supported by the FDA’s Project Optimus³⁴, where drug developers are encouraged to move away from a “no regrets” MTD-like strategy towards a more nuanced, albeit harder to develop, protocol that balances efficacy and toxicity.

It should be emphasized that AT is primarily considered in the context of incurable disease where resistant subclones are present. If no resistant cells are present in the initial tumor, then an adaptive approach could theoretically reduce the possibility of cure by maintaining a residual population of sensitive cells rather than pursuing complete eradication. However, even in a setting where control and not cure is the goal, implementation of response or toxicity-guided treatment breaks may present communication challenges for both patients and clinicians³⁵. Treatment interruptions may be interpreted as suboptimal treatment, which can make it difficult to convey that a pause is an intentional part of the therapeutic strategy. Misperceptions may also stem from a belief that more intensive or continuous therapy always yields better outcomes. For this reason, it is important that discussions with patients emphasize that pauses triggered by decreases in tumor size or by toxicity thresholds are intentional, designed to balance tolerability with maximizing long-term disease control, and should not be interpreted as therapeutic abandonment. To our knowledge, no studies have yet specifically explored patient perceptions of AT; understanding these views will be critical for clinical translation.

While beyond the scope of our present investigation, our model can be applied to a broader range of AT protocols. Like the first AT trial on castration resistant metastatic prostate cancer²¹, here we simulated therapy cycling based on fixed upper and lower thresholds of tumor burden, respectively. Mouse model experiments^36,37,38,39 and a current clinical trial on ovarian cancer⁴⁰ have used dose de-escalation as tumor burden drops and dose escalation as burden rises. Range-bounded AT lets the upper and lower bounds of on/off cycles drift upwards, taking advantage of stronger inter-cell type competition with increasing tumor burden⁴¹.

Future work could extend this model in several directions. First, threshold optimization is critical: the interplay between tumor size thresholds and toxicity thresholds is complex, and our results suggest the existence of “sweet spots” that maximize tumor control. Second, our analysis has focused on dose skipping but dose modulation is another strategy to manage toxicity. Extending our framework to incorporate both dose skipping and dose-modulation strategies will be an important next step. Third, the frequency of treatment decisions is likely to be critical: while we assumed daily monitoring, less frequent assessments (weekly or monthly) may amplify overshoot and alter outcomes, as highlighted by Gallagher et al.³². Finally, integrating more realistic toxicity measures (e.g., neutropenia-based PK/PD models or composite toxicity indices) and incorporating patient-reported outcomes will be essential for translational relevance.

In conclusion, our modeling study demonstrates how incorporating toxicity constraints into AT can reduce treatment-associated toxicity without sacrificing efficacy. By balancing patient tolerability with treatment efficacy, such toxicity-informed adaptive protocols hold the promise of turning at least some types of incurable cancers into manageable chronic conditions. This approach may extend survival rates, improve patient quality of life, and potentially bring us closer to truly personalized cancer treatment.

Methods

Mathematical Model

The model analyzed in this study integrates drug pharmacokinetics and toxicity dynamics into an existing framework¹⁶ of competitive interactions between treatment-sensitive ($S(t)$) and treatment-resistant ($R(t)$) cancer cells. The conceptual model assumes that the two subpopulations compete for, and grow logistically up to, a shared carrying capacity $K$. Sensitive cells can be killed by the drug, while resistant cells cannot. Drug concentration $C(t)$ is described by a simple one compartment pharmacokinetic model with linear clearance. Toxicity $T(t)$ increases with drug concentration $C(t)$, and resolves at a fixed linear rate.

The resulting system of equations is as follows:

$${S}^{{\prime} }\left(t\right)={\alpha }_{S}S\left(1-\frac{S+R}{K}\right)-\delta {CS},$$

(1)

$${R}^{{\prime} }\left(t\right)={\alpha }_{R}R\left(1-\frac{\beta S+R}{K}\right),$$

(2)

$${C}^{{\prime} }\left(t\right)={\rm{source}}-{\rm{\lambda }}C,$$

(3)

$${T}^{{\prime} }\left(t\right)=\mu C-\gamma T.$$

(4)

Here ${\alpha }_{S}$ and ${\alpha }_{R}$ are the growth rates of the sensitive and resistant cells, respectively, and $\beta$ is the competition suppression factor that sensitive cells impose on resistant cells. The drug is assumed to be instantaneously injected into the model through the source term, has a clearance rate of $\lambda$, and a killing efficacy of $\delta$ on the sensitive cells. Toxicity accumulates proportionally to the drug concentration with parameter $\mu$, and decays exponentially with parameter $\gamma$. Baseline values of parameters, together with their meanings, are summarized in Table 1. We remark that in our formulation, resistance to the drug incurs a cost so that resistant cells cannot proliferate faster than sensitive cells. Mathematically, we express the growth rate of resistant cells as: ${\alpha }_{R}=\epsilon {\alpha }_{S}$, where $0\le \epsilon \le 1$.

In the interest of generality and wide applicability, and because here we are not modeling any specific cancer type, drug, or experimental dataset, we rescale our model variables as follows. The tumor cell subpopulations $S(t)$ and $R(t)$ are expressed as fractions of an assumed carrying capacity of 100, making them interpretable as percentages of sensitive and resistant tumor cells. The drug concentration $C(t)$ is dimensionless, reflecting its role as a generic chemotherapeutic agent. Likewise, toxicity $T(t)$ is measured in arbitrary units to encompass a broad range of potential side effects. These rescalings provide a flexible framework within which we can explore treatment dynamics across various scenarios. The baseline parameters were selected to demonstrate the dynamical differences between the four protocols detailed in ‘Defining Treatment Protocols’ subsection.

Equations (1)–(4) are solved in MATLAB® using ode23s, a stiff solver that implements a modified Rosenbrock formula of order two. Unless otherwise indicated, the scripts used to solve and analyze the toxicity model are available at https://github.com/jgevertz/toxicity.

Defining treatment failure

Following the RECIST (Response Evaluation Criteria in Solid Tumors) criteria used in clinical practice, which typically defines disease progression as at least a 20% increase in tumor size from some baseline⁴², we define therapy failure as the total tumor burden rising 50% above its value at treatment initiation. In our model, the tumor burden starts at 50% of the carrying capacity, and treatment failure is thus defined as a tumor burden of 75% of the carrying capacity (that is, the initial tumor burden is 50, and treatment failure occurs when $S+R\ge 75$). TTP is defined as the time from treatment initiation until the tumor burden reaches this treatment failure size. We emphasize that in a clinical setting, an oncologist would not know the tumor’s carrying capacity and, instead, would make decisions based on measurable tumor burden obtained via imaging or other methodologies. To reflect this, efficacy-based decisions in our model are related to the tumor size at therapy initiation rather than to its (unknown) carrying capacity. In our numerical simulations, the tumor’s initial composition is set at 99% sensitive cells and 1% resistant cells at 50% carrying capacity, or $S\left(0\right)=49.5$ and $R\left(0\right)=0.5$. Treatment starts on day 1, so both the drug concentration and toxicity are assumed to be zero initially, that is, $C\left(0\right)=T\left(0\right)=0$.

Defining treatment protocols

Next, we present our approach to integrating both efficacy-based (tumor size) and toxicity-based (drug-induced toxicity level) feedback into the decision rules for starting, pausing, and resuming treatment. The resulting four protocols are summarized in Fig. 1.

Daily protocol

A fixed dose of 1 unit is administered at daily intervals (time $t=1$ day) until either treatment failure or the total elapsed time hits 100 days, whichever occurs first. Tumor burden is monitored daily, just before treatment administration, to check for disease progression. This protocol is summarized in Fig. 1A.

Adaptive protocol

Treatment is administered in daily 1-unit doses until the total tumor size, $S\left(t\right)+R(t)$, falls below the Rx_off threshold. At this point, therapy is paused until the tumor size regrows and again exceeds the upper Rx_on threshold, prompting treatment to resume. As with the daily protocol, tumor burden is monitored daily, just before potential treatment administration, to determine if disease progression has occurred. This protocol is summarized in Fig. 1B.

Toxicity feedback

This protocol can be incorporated into either of the previous two protocols described above, thus acting as an override switch for pausing therapy. Specifically, if toxicity levels are above a designated threshold Tox_off when the next treatment decision is to be made, then treatment is halted regardless of the tumor size. Therapy can only resume when the toxicity level at the time of the next treatment decision drops below the Tox_on threshold. In the daily protocol with toxicity feedback, dosing (re)starts when toxicity is below Tox_on, and will continue until toxicity rises above Tox_off. In the adaptive protocol with toxicity feedback, however, treatment (re)starts when both toxicity is below Tox_on and the tumor size is above Rx_on. Treatment will pause when either toxicity is above Tox_off or the total tumor size is below Rx_off. These conditions are tested daily, prior to dose administration. The cycles will continue until treatment failure or total simulation time hits 100 days. Figure 1C and D summarize adding toxicity feedback to the daily and adaptive protocols, respectively.

Each administered dose is assumed to be 1 unit of drug, and the potential treatment period is simulated over 100 days. During this time, the simulation may or may not predict disease progression by day 100. Time to progression (TTP) and tumor composition at TTP are recorded at the end of each simulation. If the tumor did not progress in the 100-day simulation, then TTP is recorded to be 150 (a randomly selected value greater than 100 to indicate treatment did not fail during the simulation time). Simulations will follow one of the four treatment protocols described above.

Global sensitivity analysis framework

To assess the robustness of model predictions to variability in parameter values, we conduct a global sensitivity analysis using the extended Fourier Amplitude Sensitivity Test (eFAST). eFAST is a variance-based decomposition technique capable of efficiently handling nonlinear models^43,44. The method quantifies the sensitivity of a model’s output to variations in input parameters, by computing both a first-order and a total-order sensitivity index. The sensitivity of a particular input, the ratio of the total variability in the output attributed to changes in that input, is found by averaging over all other inputs. Parameters with high sensitivity indices are identified as influential to the model output, while those with low sensitivity indices may be regarded as negligible. The template used for eFAST implementation can be found here: http://malthus.micro.med.umich.edu/lab/usadata/

Herein, we assess the global sensitivity of TTP across the four treatment protocols for the following model parameters over the following ranges: the sensitive cell growth rate (${0.5\le \alpha }_{S}\le 1.5$), the resistant cell growth rate relative to the sensitive cell growth rate ($0.2\le \epsilon \le 1,\,{\alpha }_{R}=\epsilon {\alpha }_{S}$), the rate of drug-induced cancer cell death ($0.5\le \delta \le 1.5$), the competition parameter ($0.5\le \beta \le 3.5$), and the toxicity recovery rate ($0.2\le \gamma \le 0.8$). The output of this sensitivity analysis then informs a set of parameter sweeps, wherein we quantify the impact that simultaneously varying two model parameters has on TTP in each of the four treatment protocols.

We remark that the rate of toxicity accumulation $\mu$ is omitted from the global sensitivity analysis in this proof-of-concept study. This is because we are not mapping toxicity to a specific biomarker or drug, so the scale of $T(t)$ is arbitrary, and decisions to initiate or suspend therapy depend only on $T(t)$ crossing the thresholds Tox_off and Tox_on (see Table 1). Moreover, the differential equation governing $T(t)$ in Eqs. (1)–(4) can be integrated to yield the following analytical solution:

$$T\left(t\right)=\mu {\int }_{0}^{t}{e}^{-\gamma \left(t-s\right)}C\left(s\right){ds}.$$

From this, it follows that $T(t)$ and the toxicity thresholds Tox_off and Tox_on can be all rescaled by $\mu$ without affecting model dynamics or decision-making criteria. Instead, we systematically vary the toxicity thresholds to elucidate their effect on treatment outcomes (see ‘Assessing Therapy Outcome Robustness to Protocol Thresholds’ subsection).

Defining virtual patient framework

To extend our numerical simulations and two-dimensional parameter sweeps to better understand variability across parameter space, we utilize a virtual population approach⁴⁵. In our computational framework, each virtual patient (VP) in the virtual population is represented by a set of parameter values related to characteristics of tumor growth and response to treatment. That is:

$${\rm{Virtual\; Patient}}=\left({\alpha }_{S},\,{\alpha }_{R},\delta ,\beta ,\gamma \right).$$

The baseline parameter values listed in Table 1 can be thought of as corresponding to an ‘average’ patient. To construct a VP, a value for each parameter in the set $\left({\alpha }_{S},\,{\alpha }_{R},\,\delta ,\,\beta ,\,\gamma \right)$ is drawn as a simple random sample from a corresponding lognormal distribution. The distribution’s shape parameters are chosen such that the peak approximately occurs at the baseline VP parameter value reported in Table 1, and the width approximately matches the range used in the global sensitivity analysis. This process is repeated 100 times to form a heterogeneous virtual population, or 100 sets of the five VP parameters.

The carrying capacity $K$, rate of drug clearance $\lambda$, and rate of toxicity build up $\mu$ are fixed across all VPs. This decision was made to preserve the consistency of key modeling assumptions and to prevent confounding of sensitivity results due to parameter interdependencies. Since tumor size at treatment initiation was set at $0.5K$ and treatment failure at $0.75K$, varying $K$ would not affect time to progression. We fixed $\mu$ because of its relation with $\gamma$ – higher toxicity build-up rates $\mu$ can be offset by correspondingly higher toxicity clearance rates $\gamma$. Likewise, the drug’s cell-killing effect can be preserved even with a high clearance rate $\lambda$ if the rate of treatment-induced cell death $\delta$ is also high.