Fig. 3: PIDAO’s ability to escape saddle points and the interpretability of PIDAO’s hyperparameters.
From: Accelerated optimization in deep learning with a proportional-integral-derivative controller
a Escaping time of different optimizers with the same X(0) = [0.5, 1] for a quadratic function \(f({X}_{1},{X}_{2})=\frac{1}{2}({X}_{1}^{2}-\epsilon {X}_{2}^{2})\). b–d Evolution trajectories and loss errors of the PIDAO with different hyperparameters (kp, ki, kd) and X(0) = [ − 6, 5.5] for the Rastrigin loss. b Select ki = kd = 0 but increasing kp. c Select kp = 10 here and kd = 0 but increasing ki. d Select kp = 10 and ki = 14 but increasing kd. The red star in the above figures denotes the initial point X(0).
