Adaptive resetting for informed search strategies and the design of non-equilibrium steady-states

Abstract

Stochastic resetting, the procedure of stopping and re-initializing random processes, has recently emerged as a powerful tool for accelerating processes ranging from queuing systems to molecular simulations. However, its usefulness is severely limited by assuming that the resetting protocol is completely decoupled from the state and age of the process that is being reset. We present a general formulation for state- and time-dependent resetting of stochastic processes, which we call adaptive resetting. This allows us to predict, using a single set of trajectories without resetting and via a simple reweighing procedure, all key observables of processes with adaptive resetting. These include the first-passage time distribution, the propagator, and the steady-state. Our formulation enables efficient exploration of informed search strategies and facilitates the prediction and design of complex non-equilibrium steady-states, eliminating the need for extensive brute-force sampling across different resetting protocols. Finally, we develop a general machine learning framework to optimize the adaptive resetting protocol for an arbitrary task beyond the current state of the art. We use it to discover efficient protocols for accelerating molecular dynamics simulations.

Introduction

Stochastic resetting has drawn significant scientific attention in recent years^1,2,3,4. In it, a stochastic process is stopped at random times and restarted with independent and identically distributed initial conditions. The ability to create with it non-equilibrium steady-states (NESS)^{1,2,3,4,5,6,7,8}, and the relative ease with which stochastic resetting is implemented in lab conditions, made it a promising guinea pig for testing novel ideas in non-equilibrium statistical physics^{9,10,11,12,13}. Moreover, stochastic resetting can expedite first-passage processes^2,14,15,16, which has proven useful in accelerating computer algorithms, such as Las Vegas algorithms^17,18, search algorithms^19,20, and molecular dynamics simulations^21,22,23. Resetting is also central to our understanding of biological phenomena such as enzymatic catalysis and inhibition^24,25, and backtrack recovery by RNA polymerases^26,27.

The power and usefulness of the theory of stochastic resetting come from its ability to predict the time evolution, steady-state, and first-passage time (FPT) distribution of a process undergoing restart, from its counterpart without resetting^2,5,14,15. However, such a general framework does not exist for cases where the resetting protocol depends on the state and history of the underlying process. This severely limits the applicability of the theory, as we further explain below.

Resetting is known to accelerate search processes under certain conditions^2,14,15,28. However, since regular resetting is agnostic to the system state, it could also occur very close to the target. Preventing such undesirable resetting events is clearly beneficial. Searching agents may do this by sensing their proximity to the target, e.g., by smell, and adapt their resetting probability in response, as illustrated in Fig. 1a. Similarly, when using resetting to expedite molecular simulations, adapting the resetting rate by including information on the reaction progress leads to substantial acceleration²⁹. All these examples fall outside the scope of the existing theory of stochastic resetting but can be addressed by accounting for a possible coupling between the state of the system and the resetting protocol. In these cases, the resetting rate adapts to the underlying dynamics in a state- and time-dependent manner, and we will thus refer to it as adaptive resetting.

Fig. 1: Use cases of adaptive resetting.

a Informed search strategies. To shorten search time, a foraging animal occasionally returns home (resetting). The animal adapts its resetting rate by gathering information on the proximity of food, e.g., by using the sense of smell. In this way, it avoids undesirable home returns, allowing it to locate food much faster. b Non-equilibrium steady-state (NESS) design via adaptive resetting. Starting from a Boltzmann distribution (upper panel), resetting to the same initial conditions at a state-independent rate results in the same steady-state (lower left panel). However, position-dependent resetting can produce nonuniform distributions, such as the curved arrow-shaped distribution in the lower right panel.

Adaptive resetting also paves the way for using resetting as a tool for engineering NESS that could not be obtained with standard resetting. It is well known that resetting leads to a steady-state, if the time between resets has a finite mean^2,5,30,31. For example, resetting a particle diffusing in one dimension at a constant rate results in a Laplace distribution steady-state¹. However, a state-dependent resetting rate could lead to a broader range of possible outcomes. Imagine, for example, a particle diffusing in a two-dimensional box whose initial position is uniformly distributed. Stochastic resetting to this distribution at a state-independent rate will again result in a uniform, featureless, NESS. However, incorporating a position-dependent resetting rate can break symmetry and allow the design of more complicated NESS, as we illustrate in Fig. 1b.

So what is the key barrier hindering the development of a theory of adaptive, state- and time-dependent, resetting? For standard resetting, the dynamics of the probability distribution without resetting is sufficient to predict the behavior under restart². For example, the probability distribution describing the position of a diffusing particle under resetting can be obtained directly from the fact that the probability distribution of a freely diffusing particle is Gaussian. As a result, the steady-state under any distribution of resetting times can also be obtained. Similarly, the FPT distribution with resetting can be directly predicted from its counterpart without resetting¹⁵. Unfortunately, this is no longer true for adaptive resetting, as the probability of reaching a certain point at a certain time without being reset depends on the entire history of the trajectory, via the state- and time-dependence of the resetting rate. In this case, one must account for all possible trajectory histories and their relative weights, which is a staggering challenge.

For overdamped Brownian motion under a conservative potential, Roldán and Gupta showed that solving a state-dependent resetting problem could be mapped to evaluating the quantum propagator of an auxiliary system with a potential that depends on the resetting protocol³². Essentially, they invoked the path integral formulation of quantum mechanics to sum over all possible trajectories. This can be done for a free particle or a quantum harmonic oscillator, where analytical solutions are known. Yet, analytical solutions for more complicated systems are notoriously hard to obtain, and numerical solutions are costly. Other authors have thus focused on specific state-dependent resetting problems, giving ad hoc solutions by circumventing the calculation of the path integral^{33,34,35,36,37,38,39,40,41,42}. This highlights the need in a general framework for stochastic processes with adaptive resetting, allowing to predict their behavior directly from trajectories without resetting, as is done for standard resetting.

Below, we present a general formulation of adaptive resetting. Practically, we show that a single set of trajectories without resetting, obtained experimentally, numerically, or analytically, is sufficient to estimate all key observables with adaptive resetting via a simple reweighing procedure: the mean FPT (MFPT), the entire FPT distribution, the propagator, and the steady-state distribution.

We first demonstrate the power of our approach by analyzing informed search strategies, which utilize environmental information to lower the resetting rate in the vicinity of the target. There, we discover a crossover in the optimal resetting strategy depending on the ratio of two length scales: the initial distance to the target and the range at which the target can be sensed effectively. Next, we use our approach to predict the tail of the NESS formed by a diffusing particle under a power-law resetting rate, which was so far out of reach theoretically. We also show how to use adaptive resetting to break the spatial symmetry of the underlying stochastic process and design complex and detail-rich NESS.

Finally, we demonstrate how our framework can be used to optimize the adaptive resetting protocol to perform a given task. We do so by representing the state-dependent resetting probability as a neural network and training it to optimize a loss function that can be any observable of the process with adaptive resetting. The loss is calculated based on trajectories without resetting using our reweighing procedure. As a concrete example, we find an optimized adaptive resetting protocol that minimizes the MFPT of conformational transitions in simulations of the mini-protein chignolin in explicit water.

Results

The MFPT with adaptive resetting

We begin by defining an adaptive, i.e., a state- and time-dependent, resetting rate r(X, t), with X being the state of the system and t representing time. Given a trajectory, $\{{{{\boldsymbol{X}}}}({t}^{{\prime} }),0\le {t}^{{\prime} }\le t\}$, we can define a random variable describing its resetting time R via its cumulative distribution function

$$\Pr (R\le t)=1-\exp \left(-\int_{0}^{t}r({{{\boldsymbol{X}}}}({t}^{{\prime} }),{t}^{{\prime} })\,d{t}^{{\prime} }\right).$$

(1)

Note that R distributes differently for different trajectories under the same functional choice of the resetting rate. We emphasize that any state- and time-dependent resetting strategy can be represented in this way. For example, for an exponential resetting time, one takes a constant resetting rate r(X, t)  =  r. More generally, to represent a general resetting time distribution whose survival function is Ψ(t)  =  1 − Pr(R ≤ t), one takes $r({{\boldsymbol{X}}},t)=-\frac{d}{dt}\,{\mbox{ln}}\,[\Psi (t)]$, which does not depdend on X. Similarly, one may consider an arbitrary spatial dependence of the resetting rate, e.g., r(X, t)  = rΘ(X), where Θ(X) is the Heaviside step function. More generally, arbitrary state-time couplings can also be captured as no restrictions are stipulated on r(X, t) other than it being non-negative. For simplicity of demonstration, in concrete examples given in this paper, we take resetting rates r(X) which are time-independent, although the theory developed below is completely general.

We next construct the random variable describing the FPT under the resetting protocol r(X, t). We observe that a first-passage process with resetting can be described in the following way: First, a trajectory is sampled, and this determines T, i.e., the FPT without resetting. The trajectory also sets the distribution of R via Equation (1). We next sample R, and if T≤R, we conclude that the FPT is simply T. If, however, R < T, resetting occurs before first-passage, a new trajectory is sampled, and the procedure is repeated, tallying R. Overall, the random variable describing the FPT under resetting is

$${T}_{R}=\left\{\begin{array}{ll}T \hfill \quad &\,{{\mbox{if}}}\,T\le R,\\ R+{T}_{R}^{\prime }\quad &\,{{\mbox{if}}}\,R < T,\end{array}\right.$$

(2)

where ${T}_{R}^{{\prime} }$ is an independent and identically distributed copy of T_R. Using the total expectation theorem, and averaging over all the trajectories, and all realizations of R, we find that the MFPT under restart is

$$\langle {T}_{R}\rangle=\left(\frac{1}{\Pr (T\le R)}-1\right)\langle R| R < T\rangle+\langle T| T\le R\rangle .$$

(3)

From here, it is clear that the MFPT under restart can also be written as $\langle {T}_{R}\rangle=\frac{\langle min(T,R)\rangle }{\Pr (T\le R)}$. We note that the same result was previously obtained for state-independent resetting using a similar technique^14,15. A detailed derivation of Equation (3) is given in section 1 of the Supplementary Information.

We can interpret Equation (3) as illustrated in Fig. 2. For a given trajectory with resetting, the first passage is composed of several failed attempts to complete the process, each ending in resetting. These are followed by one successful attempt, ending in first-passage, and completing the process. The total number of attempts, failed plus successful, is geometrically distributed. Namely, all attempts are statistically independent with a success probability Pr(T ≤ R). Therefore, the average number of attempts is 1/Pr(T ≤ R). The mean duration of an unsuccessful attempt is 〈R∣R < T〉, and the mean duration of a successful one is 〈T∣T ≤ R〉. The MFPT with resetting is just the sum of the mean duration of a failed attempt times the mean number of failed attempts, with the mean duration of the final successful attempt.

Fig. 2: An illustration of trajectories with resetting.

Each trajectory is composed of segments ending in resetting (yellow frame) and a final segment ending in first-passage to the gray dashed line (blue frame). The mean first-passage time under restart 〈T_R〉 is given by Equation (3). It is the mean length of the blue segments 〈T∣T ≤ R〉, plus the mean length of the yellow segments 〈R∣R  <  T〉, multiplied by the average number of yellow segments, 1/Pr(T ≤ R) − 1, in a single trajectory.

Equation (3) is the first key result of this paper. It generalizes known results for the MFPT under resetting for a much broader class of resetting protocols. Usually, one assumes that the first passage and resetting processes are completely decoupled, making T and R statistically independent^2,14,15. Here, we allowed the resetting rate to explicitly depend on the trajectory, making T and R statistically dependent, but showed that the same equations hold. The case of regular resetting is obtained trivially by omitting the state dependence from the resetting rate in Equation (1).

Estimating the MFPT from trajectories without resetting

We now want to estimate how different state- and time-dependent resetting protocols affect the MFPT. Since analytical expressions are unavailable in almost all cases, one approach is to directly sample trajectories with resetting and estimate the MFPT. If one is interested in a single resetting protocol, that may be the most efficient approach. However, if one would like to explore a wide range of resetting protocols, e.g., for design and optimization purposes, this approach becomes impractical. Each resetting protocol would require resampling the trajectories from scratch, which is very costly. We instead propose a much more efficient approach, based on Equation (3). Importantly, we show that sampling a single set of trajectories with no resetting is enough to estimate the MFPT for any state- and time-dependent resetting protocol through a reweighing procedure.

We begin with a set of N independent and identically distributed initial conditions, from which we generate trajectories X ⁱ(t) ending in first-passage at a random time T_i, where i  = 1, 2, …N. We stress that any form of initial condition could be used, e.g., a delta function for positions and Maxwell-Boltzmann distribution for velocities. We decompose each trajectory into n_i time steps of length Δt, such that T_i  =  n_iΔt. For Δt that is sufficiently short, the probability of resetting the i − th trajectory at the j − th time step, given that no resetting occurred previously, is ${p}_{j}^{i}=r({{{{\boldsymbol{X}}}}}^{i}(\;j\Delta t),j\Delta t)\Delta t$. Then, we define the survival probability of trajectory i up to time step j, i.e., the probability that no resetting occurred prior to time step j, as

$${\Psi }_{j}^{i}={\prod}_{k=1}^{j-1}\left(1-{p}_{k}^{i}\right).$$

(4)

Using the survival probability, we can estimate the success probability as

$$\Pr (T\le R)\approx \frac{1}{N} {\sum}_{i=1}^{N}{{\Psi }_{{n}_{i}}^{i}},$$

(5)

and the mean duration of a successful segment ending in first-passage as

$$\langle T| T\le R\rangle \approx \frac{1}{N\Pr (T\le R)}{\sum }_{i=1}^{N}{\Psi }_{{n}_{i}}^{i}{n}_{i}\Delta t.$$

(6)

Equation (5) is simply the average probability of surviving the entire trajectory without resetting. Equation (6) evaluates the average trajectory duration reweighed by its survival probability, as the probability of sampling trajectory i given that T ≤ R is ${\Psi }_{{n}_{i}}^{i}/[N\Pr (T\le R)]$. Similarly, we can estimate the mean duration of a failed segment ending in resetting as

$$\langle R| R < T\rangle \approx \frac{1}{N(1-\Pr (T\le R))}\mathop{\sum}_{i=1}^{N} {\sum}_{j=1}^{{n}_{i}-1}{\Psi }_{j}^{i}\,{p}_{j}^{i}\,j\Delta t,$$

(7)

where ${\Psi }_{j}^{i}\,{p}_{j}^{i}$ is the probability that trajectory i survives without resetting  j − 1 steps, and is reset exactly at time step j. Substituting Equations (5–7) into Equation (3), we obtain an estimation for the MFPT whose accuracy is limited only by the sample size N and the time step Δt.

In section 2 of the Supplementary Information and in Supplementary Fig. S1, we benchmark this procedure by comparing its result with the analytical MFPT for diffusion with asymmetric resetting, obtained in ref. ³⁴. We find excellent agreement between them, using sets of only N  =  10⁴ trajectories without resetting from numerical simulations. The estimated MFPT values are up to  ~3% off the analytic solution in almost all cases, for resetting rates in a range spanning over three orders of magnitude. Full simulation details are given in section 3 of the Supplementary Information.

Search with environmental information

We now employ our framework to study a biologically motivated example. Imagine a diffusive searcher in one dimension, e.g., a foraging animal, with a diffusion coefficient D. The searcher starts at L, and the target is located at the origin. Without resetting, the MFPT for this search process diverges⁴³. However, Evans and Majumdar showed that using standard resetting, the MFPT becomes finite¹. Recent studies suggest that animal homing might be a manifestation of this observation^28,44. As illustrated in Fig. 1a, the searcher may further reduce the MFPT by incorporating information about the distance from the target, such as chemical and olfactory signals, avoiding resetting too close to it. To model this environmental information, we employ a position-dependent resetting rate that increases up to a maximal value r₀ as a function of the distance from the target,

$$r(x)={r}_{0}\left(1-\frac{1}{1+{\left(x/b\right)}^{2}}\right),$$

(8)

where b is the typical length scale at which environmental information affects the searcher.

Figure 3a presents the MFPT as a function of the control parameters r₀ and b. We observe that there are two regimes. In the limit of b ≪ L, there is no information effectively, and the behavior is like free diffusion with standard resetting at a constant rate r₀. The minimum of the MFPT is obtained at the expected rate for free diffusion¹, indicated by the black dashed line. However, when b ≫ L, Equation (8) is approximated by r(x) ≃ r₀x²/b², and resetting is governed by a single parameter r₀/b² that determines the curvature of the parabolic resetting rate. When the curvature is very small, the problem asymptotically converges to free diffusion, where the MFPT diverges. When the curvature is very large, the particle is trapped in the vicinity of L, and the MFPT again diverges. Therefore, we expect that there will be an optimal value of the curvature, but there is currently no analytical expression for it. Using our method, we find the optimal value that leads to the highest acceleration, r₀/b² ≃ 5.6 (black dotted line), which results in an MFPT of ≃1.

Fig. 3: Search with environmental information.

a The mean first-passage time to the origin as a function of r₀ and b for one-dimensional diffusion with diffusion coefficient D  =  1. Motion starts at L  =  1 and is conducted under position-dependent resetting of the form given in Equation (8). The dashed and dotted lines indicate the optimal rate for diffusion with a constant resetting rate, r₀  =  2.5, and the optimal rate for a parabolic resetting, r₀  =  5.6b², respectively. b First-passage time distributions ${f}_{{T}_{R}}(t)$ for the three marked points in the phase space of (a), estimated from simulations without resetting (solid lines), and brute-force simulations with resetting (markers). For simulations with resetting, we plot the mean $\pm 1/\sqrt{10}$ standard deviation of ten independent repetitions with 10⁴ trajectories each (errors are smaller than the marker size in almost all cases).

We emphasize that all the results of Fig. 3a were obtained using a single set of trajectories with no resetting and simply re-evaluating Equations (4–7) for every value of r₀ and b, leading to a different MFPT through Equation (3). Previously, obtaining the above results would have required performing an ensemble of brute-force simulations at every value of the parameters r₀ and b, to map out the MFPT phase space. This approach would be computationally prohibitive in many cases. Moreover, the strength of our approach becomes even clearer when considering other forms of r(x) apart from Equation (8). Tackling these using brute-force simulations, analytical methods, or experiments, would require a complete reanalysis of the problem. On the other hand, using our method, we can use the same initial ensemble of trajectories to generate the MFPT phase space for any resetting protocol, with minimal added cost. In section 4 of the Supplementary Information, we demonstrate this for several other sigmoidal-shaped resetting protocols (Supplementary Fig. S2).

By efficiently extracting the MFPT phase space for several functional forms of the resetting rate [Equations (S5–S8) of the Supplementary Information], we identify an interesting trend. For all forms, there are two regimes. The behavior at b ≪ L mimics that of diffusion with a constant resetting rate, and the behavior at b ≫ L is determined entirely by the leading-order term in the Taylor expansion of the resetting rate near the origin. All other particularities of the resetting rate only play a role at the crossover regime b ≈ L.

Finally, we are not limited to the MFPT and can obtain the full FPT distribution for any value of r₀ and b using the same single set of trajectories. Figure 3b shows the FPT distributions obtained for the three values marked in Fig. 3a, as estimated using the set of trajectories without resetting (solid lines), and from direct simulations with resetting for comparison (markers). We find excellent agreement between the two for all cases. The derivation of how to estimate the FPT distribution from trajectories without resetting is given in section 5 of the Supplementary Information.

Estimating the propagator and NESS

We next show how to evaluate the full propagator of the process under state- and time-dependent resetting from a single set of trajectories without resetting. The long-time limit of the propagator provides the NESS achieved by the resetting protocol.

We begin by defining the propagator, ${G}_{j}(\overrightarrow{x})$, as the probability to be at $\overrightarrow{x}$ at time step j. It has two contributions, the first from trajectories that are at $\overrightarrow{x}$ in time step j without undergoing resetting before. The second contribution is from trajectories that underwent resetting at least once. Overall, the propagator can be written as,

$${G}_{j}(\overrightarrow{x})={G}_{j}^{\Psi }(\overrightarrow{x})+ {\sum}_{k=1}^{j-1}{P}_{k}{G}_{j-k}(\overrightarrow{x}),$$

(9)

where ${G}_{j}^{\Psi }(\overrightarrow{x})$ is the probability of being at position $\overrightarrow{x}$ at time step j without undergoing resetting along the way. The second term considers scenarios in which trajectories were reset for the first time at time step k, and j − k steps later are at $\overrightarrow{x}$. Here, P_k is the probability of undergoing resetting for the first time at time step k.

To solve Equation (9), we write it in matrix form and rearrange, to get

$$\overrightarrow{G}(\overrightarrow{x})={\left({{{\boldsymbol{I}}}}-{{{\boldsymbol{P}}}}\right)}^{-1}{\overrightarrow{G}}^{\Psi }(\overrightarrow{x}),$$

(10)

where the matrix elements of P are given by P_i,j ≡ P_i−j if i  >  j and zero otherwise. Thus, to estimate the propagator under state- and time-dependent resetting, we only need to estimate P and ${\overrightarrow{G}}^{\Psi }(x)$ from trajectories without resetting. We do this as follows:

$${{{{\boldsymbol{P}}}}}_{i,j}=\left\{\begin{array}{ll}{P}_{i-j}=\langle {\Psi }_{i-j}{p}_{i-j}\rangle \approx \frac{1}{N}{\sum }_{n=1}^{N}{\Psi }_{i-j}^{n}\,{p}_{i-j}^{n} &\,{{\mbox{if}}}\,i > j,\hfill \\ 0 \hfill &\,{{\mbox{otherwise}}},\end{array}\right.$$

(11)

where we recall the definitions of Ψ_i−j and p_i−j in and above Equation (4), and sum over trajectories. Similarly, we have

$${G}_{j}^{\Psi }(\overrightarrow{x})\approx \frac{1}{N} {\sum}_{i=1}^{N}{\Psi }_{j}^{i}\delta (\overrightarrow{x}-{{{{\boldsymbol{X}}}}}^{i}(\;j\Delta t)).$$

(12)

The advantage of this approach is that the solution is obtained in real-time, however, for long times, it requires inverting very large matrices.

Alternatively, the long-time behavior and the NESS can be obtained more efficiently by taking the Z-transform of Equation (9) and using the convolution theorem. This gives

$$\hat{G}(\overrightarrow{x},z)=\frac{{\hat{G}}^{\Psi }(\overrightarrow{x},z)}{1-\hat{P}(z)},$$

(13)

where $\hat{f}(z)={\sum }_{j=0}^{\infty }{f}_{j}{z}^{j}$ is the Z-transform of the series {f₀, f₁, f₂, . . }. While Equation (13) is given in discrete time, an equivalent equation using the Laplace transform is valid for continuous time. We note that a special case of the continuous-time analog of Equation (13), for an overdamped particle diffusing in a potential and space-dependent resetting rate, was given by ref. ³². Our work extends this result to a general stochastic process.

The final value theorem for Z-transforms states that ${G}_{NESS}(\overrightarrow{x})={\lim }_{z\to {1}^{-}}(1-z)\hat{G}(\overrightarrow{x},z)$. By using it, we get (see section 6 of the Supplementary Information)

$${G}_{NESS}(\overrightarrow{x})=\frac{{\hat{G}}^{\Psi }(\overrightarrow{x},1)}{\langle {N}_{R}\rangle },$$

(14)

where 〈N_R〉 is the mean number of time steps between consecutive resetting events. Note that the steady-state in Equation (14) is well defined whenever 〈N_R〉 is finite, regardless of whether or not the process without resetting has a steady-state. This is a generalization of a well-known result in the theory of standard resetting to state- and time-dependent resetting^2,5,30,31.

To estimate the NESS, we first sample a set of N trajectories without resetting of length MΔt. We stress that M should be large enough such that, had we used resetting, the probability of surviving M steps without resetting would be negligible, i.e., ${\Psi }_{M}^{i}\ll 1\,\forall i$. Then, we use Equations (12) and (14), and the definition of the Z-transform, to obtain

$${G}_{NESS}(\overrightarrow{x})\propto {\hat{G}}^{\Psi }(\overrightarrow{x},1)=\mathop{\sum }_{j=1}^{\infty }{G}_{j}^{\Psi }(\overrightarrow{x})\approx \frac{1}{N}\mathop{\sum}_{j=1}^{M} {\sum}_{i=1}^{N}{\Psi }_{j}^{i}\delta (\overrightarrow{x}-{{{{\boldsymbol{X}}}}}^{i}(\;j\Delta t)).$$

(15)

This estimation results in an unnormalized distribution, which should be normalized. The normalization factor provides an estimate for the mean time between consecutive resetting events, 〈N_R〉. Equation (15) shows that the estimation of the NESS with resetting, from trajectories without resetting, is done by averaging the histogram of positions over time and trajectories, but reweighing each trajectory, at every time step, by its survival probability.

Prediction and design of non-equilibrium steady-states

The above results can be used to predict and design NESS of spatially-dependent resetting protocols. We demonstrate this using two examples.

It is well known that for free diffusion with a constant resetting rate, a Laplace distributed NESS emerges¹. An analytical solution for the NESS of diffusion with a parabolic resetting rate r(x)  =  r₀x² is also known³². Interestingly, in both cases, the tails of the NESS decay as $\sim {e}^{-| x{| }^{\alpha }}$, with α  =  1 for the constant resetting rate, and α  =  2 for the parabolic resetting rate. This raises a more general question: what is the asymptotics of the NESS for diffusion with a power-law resetting rate r(x)  = r₀∣x∣^λ. While there are currently no known closed-form solutions for the NESS with λ  ≠  {0, 2}, we can easily estimate the resulting NESS using the procedure described in the previous section.

Figure 4a presents the NESS for diffusion with a resetting rate r(x)  =  r₀∣x∣^λ, for λ  = {0, 1, 2, 3}. Red solid lines indicate the analytical solution when it is known. When analytical solutions are not known, we present results of brute-force Langevin dynamics simulations with stochastic resetting (yellow solid lines) instead. The blue dashed lines are obtained using the estimation method developed in the previous section, based on a single set of free diffusion trajectories without resetting. In all cases, we observe a good agreement between our method and the ground truth (theory or brute-force simulations).

Fig. 4: Non-equilibrium steady-state of free diffusion under adaptive resetting.

Non-equilibrium steady-state for one-dimensional diffusion with a resetting rate r(x)  =  r₀x^λ, for r₀  =  1 and $\lambda=\left\{0,1,2,3\right\}$: a The probability distribution as a function of x on a semi-log scale. b The negative logarithm of the probability distribution on a log-log scale. Analytic solutions are given in red, results from brute-force simulations in yellow, and estimations based on trajectories with no resetting in blue. The black dotted lines in (b) give linear fits in the region x > 3.25.

In Fig. 4b, we show that the NESS asymptotics obeys $\sim {e}^{-| x{| }^{\alpha }}$ and estimate the values of α through linear regression. This procedure retrieves the correct values for λ  =  {0, 2}, where we estimate α  ≃  {1.04, 2.00}, respectively. For λ  =  {1, 3}, we estimate α ≃ {1.57, 2.53}, respectively. Following these results, we hypothesize that for a resetting rate r(x) ~ ∣x∣^λ, the tails of the NESS decay as $\sim {e}^{-| x{| }^{\frac{\lambda }{2}+1}}$. While this hypothesis remains to be proven, it exemplifies the power of our approach, revealing new phenomena and inspiring future work.

Next, we demonstrate how to use our approach to engineer desirable NESS of interest. We do so by designing the arrow-shaped steady-state, that is illustrated in Fig. 1b. We start with a system composed of a diffusing particle within a box with reflecting boundaries at x  =  {−5, 5} and y  = {−5, 5}. Restart takes the particle to a uniformly distributed initial position within the box. The resulting NESS, for a position-independent resetting rate, is uniform. We seek to design a position-dependent resetting rate that will generate the desired arrow-shaped NESS, using only a single set of trajectories without resetting.

A naive approach would be to simply apply a constant resetting rate within the desired shape and zero resetting rate outside (Equation (S15) in the Supplementary Information). However, using our approach to predict the resulting steady-state, we find that this naive guess leads to a very fuzzy distribution, in which the arrow is barely discernible (see Fig. 5a). This problem cannot be solved by increasing the resetting rate. The reason is that, for every reasonably complicated NESS, there will be areas that are very hard to reach without crossing regions with a high resetting rate. In our case, the area engulfed by the arrow can only be approached from the right, which significantly lowers its occupation probability, leading to the fuzzy arrow.

Fig. 5: Non-equilibrium steady-state design.

Two-dimensional arrow-shaped non-equilibrium steady-state (NESS). a Predicted from simulations without resetting with a naive r(x, y). b Predicted from simulations without resetting with an improved r(x, y). c Obtained from brute-force simulations with the improved resetting protocol.

Previously, testing different resetting protocols, in a trial-and-error fashion, would have required running thousands of trajectories with resetting for every protocol until the desired steady-state would have been obtained. Instead, using our approach, we can design an improved adaptive resetting protocol (Equation (S16) in the Supplementary Information) that would lead to the desired shape (Fig. 5b), using the same set of trajectories without resetting that is already available. In the improved resetting strategy, the resetting rate was increased quadratically in the distance from the center of the box, and to the right of the arrow, to prevent the accumulation of density in the box’s corners and to the right of the arrow, respectively. Results from simulations with this resetting protocol are given in Fig. 5c for comparison, showing excellent agreement with our prediction.

Neural network optimization of the resetting protocol

To conclude this paper, we show how to systematically optimize an adaptive resetting protocol for an arbitrary task based on our theory. Recently, ref. ⁴⁵ used reinforcement learning to learn resetting protocols for target search problems. In contrast, we do so by representing the adaptive resetting probability $p\left({{{\boldsymbol{X}}}},t\right)=r({{{\boldsymbol{X}}}},t)\Delta t$ as a neural network. Essentially, the input to the network is the system state and time, and the output is the probability to perform resetting. The state of the system can be represented using a set of descriptors as commonly done in machine learning applications in molecular simulations^46,47,48. Then, we define a loss function for training, which can be any observable of the process with the adaptive resetting protocol given by the neural network. For example, this could be any function of the MFPT, FPT distribution, propagator, and NESS. The training data is the trajectories without resetting, and the value of the loss function at every epoch is given by the predicted value of the desired observable under resetting using our theory. Optimization is carried out using standard techniques, such as stochastic gradient descent (see section 8 of the Supplementary Information for the full details). We demonstrate the usefulness of our machine learning framework to optimize the adaptive resetting probability for minimizing the MFPT of conformational transitions of a protein in molecular dynamics simulations.

Molecular dynamics simulations are a powerful tool, but their accessible timescales are limited to a few microseconds. Therefore, to simulate any physical phenomena on longer timescales, e.g., protein folding and crystal nucleation, one must use enhanced sampling methods^{49,50,51,52,53,54,55,56}. Stochastic resetting recently emerged as a promising technique for that purpose, either as a standalone method^21,23 or in combination with Metadynamics^22,29—a popular enhanced sampling tool^54,55. Resetting accelerated rare events in molecular dynamics and Metadynamics simulations by more than an order of magnitude, and provided an accurate inference of the kinetics of the unperturbed process. However, in almost all cases, the resetting rate employed was state-independent. We recently showed that even a very limited functional form of adaptive resetting, designed ad hoc, substantially lowered the MFPT²⁹. We now show that representing the rate by a flexible neural network allows automatic optimization and yields new adaptive resetting protocols that result in higher speedups than previously possible.

We will demonstrate our approach by finding the resetting strategy that leads to the highest speedup (lowest MFPT) for the folding of the chignolin mini-protein in explicit water, which consists of 5889 atoms (simulation details are given in section 9 of the Supplementary Information). The system has three metastable states: an unfolded state, a misfolded state, and the folded, native state. Representative configurations of the states are given in Fig. 6a. We identify the states using the C-alpha root-mean-square deviation (RMSD) from the folded configuration. The free energy along this degree of freedom is plotted in Fig. 6b. The blue and yellow stars mark the RMSD values of the unfolded and misfolded configurations of Fig. 6a, respectively. We observe that there is an energy barrier the system has to cross in order to get to the native state (the deep well around RMSD  <1.5 Å). There is no substantial barrier between the misfolded and unfolded states, such that multiple misfolding-unfolding events often occur before a successful folding. We consider two first-passage processes leading to the folded state from two initial configurations: the unfolded and misfolded states.

Fig. 6: Neural network optimization of the resetting protocol.

a Ball-and-stick representation of the folded (left), misfolded (center), and unfolded (right) states of chignolin in explicit water solvent (5889 atoms), including a cartoon representation of the backbone in crimson. The white, gray, blue, and red spheres represent hydrogen, carbon, nitrogen, and oxygen atoms, respectively. b The free energy along the C-alpha root-mean-square deviation (RMSD) from the folded configuration. The RMSD values at the unfolded and misfolded states are indicated with blue and yellow stars, respectively. c The predicted speedup under a neural network-based resetting protocol as a function of the number of epochs. The dotted and dashed vertical lines highlight values of 25 and 100 epochs, respectively. d Neural network-based resetting probability as a function of the RMSD. Dotted and dashed curves give the results obtained after 25 and 100 epochs, respectively. Results in (c, d) are given for simulations initiated at the unfolded state (blue and purple) or the misfolded state (yellow and orange).

To optimize the resetting strategy, we built a network with a simple architecture: It receives three inputs (the RMSD from the folded configuration, the radius of gyration, and the end-to-end distance of the protein) and has two inner layers with ten nodes each. The output is the probability to reset given the three inputs, restricted to be between zero and one, using a sigmoid activation function in the final layer. The network is trained by going over data of trajectories with no resetting and estimating the MFPT under the resetting probability represented by the network using Eqs. (3–7). The estimated MFPT serves as a loss function that is minimized during training. An example script to train the model is given on GitHub⁵⁷.

Figure 6c shows the training curves of the model as a function of the number of epochs when starting from the unfolded (blue) or misfolded (yellow) states. The plotted value is the estimated speedup, which is defined as the ratio of the MFPT values without and with resetting, respectively. For both curves, there is a plateau around 25 epochs (highlighted with a dotted vertical line), where the algorithm reached a local minimum. The dotted lines in Fig. 6d show the resetting protocol represented by the network at this epoch as a function of the RMSD, averaged over the other two degrees of freedom. Purple and orange colors represent simulations initiated at the unfolded and misfolded states, respectively. In both cases, the networks suggest state-independent resetting probabilities. Remarkably, these are the resetting probabilities we identified as providing the highest acceleration for state-independent resetting in refs. ^22,29.

In later stages of training, the networks find better, state-dependent resetting protocols, reaching a second plateau around the 100th epoch, highlighted with a dashed vertical line in Fig. 6c. The corresponding adaptive resetting probabilities are plotted with dashed lines in Fig. 6d. For simulations initiated at the misfolded state, the predicted speedup is very close to the one obtained in ref. ²⁹ using an ad hoc, naive functional form of a step function. The resetting probability represented by the network has a similar functional form, starting with a small probability that increases sharply to nearly  ~1 for RMSD values larger than  ~6 Ų⁹. Most importantly, for simulations initiated at the unfolded configuration, the naive step function form did not lead to any speedup, but our machine learning optimization framework finds a protocol that grows gradually with the RMSD and leads to higher speedups.

Discussion

To conclude, we presented a general formulation of state- and time-dependent stochastic resetting protocols, which we call adaptive resetting. Our formulation generalizes key results from the theory of resetting for the MFPT, FPT distribution, propagator, and NESS. We presented a numerical scheme to predict all these fundamental properties using a single set of trajectories without resetting. We focused on resetting protocols that depend only on the current state and age of the system. However, our results apply without change to resetting protocols that depend on previous states and even the entire history of the trajectories. We demonstrated the power of our approach through several examples, ranging from investigating informed search strategies to predicting and designing NESS.

We also developed a machine learning framework to optimize adaptive resetting strategies for an arbitrary task for the first time. We showed that our scheme can discover non-trivial resetting protocols for accelerating molecular simulations (lowering the MFPT) beyond the previous state of the art of state-independent resetting. In the future, the same approach can be used to optimize other observables, such as the entire FPT distribution or the NESS.

A key strength of the theory presented here is that it directly connects the statistical properties of the underlying process without resetting to those of the process with adaptive resetting. This feature enables quantitative predictions even when the dynamical equations governing the process are unknown, as demonstrated in our application to chignolin folding in Metadynamics simulations. There, the RMSD collective variable exhibits non-diffusive, history-dependent dynamics, yet our framework could still be applied. Such capabilities are crucial when working with real-world data from experiments or simulations. This general framework also opens the door to discovering universal properties of adaptive resetting processes. For instance, we have shown that a finite mean waiting time between resets ensures convergence to a steady-state, generalizing a known result from the theory of state-independent resetting^2,5,30,31. Future extensions of this approach may uncover further universalities in adaptive resetting.

With the broader capabilities that adaptive resetting offers, we anticipate it will advance research on core problems in enhanced sampling. In particular, we anticipate that adaptive resetting will be harnessed for the accurate evaluation of equilibrium rates and free-energies from non-equilibrium protocols^{53,58,59,60,61,62,63}. Similarly, we expect it will soon be brought into play for the estimation of rates and free energies in the NESS^64,65,66 and for investigating first-passage dynamics out of equilibrium⁶⁷. Overall, we believe that adaptive resetting will emerge as a powerful and versatile tool, opening new avenues for research and applications in non-equilibrium statistical mechanics, molecular simulations, and beyond.

Methods

Numerical simulation details are given in sections 3 and 9 of the Supplementary Information.