Adaptive Dynamic Programming in Nonlinear Control Systems

Adaptive Dynamic Programming (ADP) has emerged as a powerful framework for the design of optimal control policies in nonlinear dynamical systems, particularly when precise mathematical models are unavailable or suffer from uncertainties. By approximating the solution of the Hamilton–Jacobi–Bellman equation through iterative policy and value updates, ADP leverages real-time data to refine control laws with minimal reliance on a priori knowledge. Neural network approximators, often partitioned into critic and actor roles, facilitate the learning of value functions and control policies, ensuring convergence toward near-optimal performance. Recent advances have extended the ADP paradigm to encompass fault tolerance, sliding-mode robustness, event-triggered and self-triggered mechanisms, thereby reducing computational load and communication demands without compromising stability. These capabilities have been demonstrated across a broad spectrum of applications—from hydraulic servo actuators and spacecraft attitude control to energy systems and autonomous platforms—highlighting the global significance of ADP for real-world nonlinear control problems.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Recent studies have demonstrated the versatility of ADP for online fault-tolerant control in highly nonlinear systems. One investigation developed a data-driven ADP approach to achieve asymptotic tracking and fault compensation in hydraulic servo actuators, solving discrete-time Riccati equations iteratively without prior knowledge of system dynamics or unmeasurable states. In a complementary line of work, a self-triggered neuro-control scheme applied ADP to approximate the Hamilton–Jacobi–Bellman solution through a critic neural network with nested weight updates, introducing event-free communication instants to curtail computational and energy consumption while guaranteeing uniformly ultimately bounded stability. Another notable contribution integrated robust ADP with sliding-mode control to address mismatched uncertainties and actuator faults in cascade nonlinear systems; by designing an optimal sliding surface offline and coupling it with adaptive fault compensation, this framework secured robust stability properties and was validated on spacecraft attitude control scenarios. Collectively, these developments underscore the progression of ADP methods toward practical, resource-aware solutions for uncertain nonlinear control challenges.

Adaptive Dynamic Programming in Nonlinear Control Systems publication trend

The graph below shows the total number of articles in adaptive dynamic programming in nonlinear control systems across all publications each year (not limited to Nature Index journals).

Technical terms

Adaptive Dynamic Programming (ADP): A learning-based control methodology that iteratively approximates optimal policies by solving the Hamilton–Jacobi–Bellman equation using real-time data.

Hamilton–Jacobi–Bellman (HJB) equation: A partial differential equation whose solution yields the value function for optimal control, establishing the principle of optimality.

Critic neural network: A neural function approximator within ADP that estimates the system’s value function or cost-to-go for a given policy.

Sliding-mode control: A robust control technique that drives system trajectories onto a predefined sliding surface, ensuring insensitivity to certain classes of disturbances and uncertainties.

Self-triggered control: A strategy in which the control law determines its own update instants based on system state, reducing unnecessary computations and communications.

References

1. Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming. Mathematical Modelling and Control (2023).
2. Self-Triggered Approximate Optimal Neuro-Control for Nonlinear Systems Through Adaptive Dynamic Programming. IEEE Transactions on Neural Networks and Learning Systems (2024).
3. Robust ADP-Based Sliding-Mode Fault-Tolerant Control for Nonlinear Systems with Application to Spacecraft. Applied Sciences (2022).