Adaptive Optimal Control in Dynamic Systems

Adaptive optimal control in dynamic systems merges the principles of adaptation and optimality to regulate systems whose behaviour or environment evolve over time. At its core, this field addresses the challenge of designing controllers that learn or update their policies in real time while minimising a predefined cost functional. Classical methods rely on solving algebraic Riccati equations for linear quadratic regulator problems, but modern approaches extend to stochastic, nonlinear and time-varying contexts without full knowledge of system models. Techniques such as adaptive dynamic programming and reinforcement learning enable data-driven policy iteration, allowing controllers to converge towards optimal gains through measured input-output trajectories. Robustness to disturbances, time-varying parameters and model uncertainties is achieved by embedding learning mechanisms within the feedback loop, ensuring stability and performance guarantees. Applications span power electronics, autonomous vehicles, robotics and aerospace, where the capacity to adapt optimally in the face of unmodelled dynamics or external perturbations can yield significant gains in efficiency, safety and resilience.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Researchers have advanced data-driven policy iteration algorithms to handle continuous-time stochastic systems with multiplicative noise and Markovian switching, proving convergence of decomposed Riccati equations and demonstrating online learning of optimal control laws from subsystem data. An innovative adaptive dynamic programming framework for trajectory tracking incorporates a parametrised Q-function that explicitly models time-varying reference signals; once learned, the controller achieves zero-error tracking for linear quadratic scenarios without further tuning and outperforms traditional methods in simulation studies. Further progress in reinforcement learning for discrete-time systems has introduced an online optimal tracking scheme that augments state vectors with integral error dynamics. This results in quadratic value functions amenable to Bellman equation formulations, enabling model-free solution of algebraic Riccati equations and guaranteeing zero steady-state error under mild excitation conditions.

Adaptive Optimal Control in Dynamic Systems publication trend

The graph below shows the total number of articles in adaptive optimal control in dynamic systems across all publications each year (not limited to Nature Index journals).

Technical terms

Adaptive optimal control: A strategy that iteratively refines control policies to achieve the best performance under uncertain or changing dynamics.
Dynamic programming: An optimisation framework that solves complex problems by decomposing them into simpler subproblems via Bellman equations.
Reinforcement learning: A data-driven approach where an agent learns optimal actions through interaction and reward feedback.
Riccati equation: A matrix equation central to linear quadratic regulator problems whose solution yields optimal state-feedback gains.
Q-function: In optimal control, a function that quantifies the expected cost of executing a specific action in a given state under a particular policy.

References

1. Data‐driven policy iteration algorithm for optimal control of continuous‐time Itô stochastic systems with Markovian jumps. IET Control Theory and Applications (2016).
2. Adaptive dynamic programming for model‐free tracking of trajectories with time‐varying parameters. International Journal of Adaptive Control and Signal Processing (2020).
3. Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning. International Journal of Adaptive Control and Signal Processing (2020).