Fig. 1: Principle of homodyne gradient extraction (HGE).
From: Gradient descent in materia through homodyne gradient extraction
a (1) Small sinusoidal signals (with frequencies [f1, …, fM], amplitudes [α1, …, αM], and phases [φ1, …, φM]) are added to the parameters w. The output h(z, w + δw) (red curve) is affected by each distinct sinusoidal perturbation δw. (2) The in-phase and quadrature components (X and Y) of the modulation of h for each frequency fm are recovered by using the principle of homodyne (lock-in) detection. These magnitudes are divided by the amplitudes of the corresponding perturbations (vector α), extracting the derivatives of the output with respect to each parameter, \(\left[\partial h/\partial {w}_{1}^{\left(k\right)},\cdots,\partial h/\partial {w}_{M}^{\left(k\right)}\right]\), in iteration step k. (3) The approximated gradient in step k along with the derivative of the loss with respect to the output current, \(\partial E/\partial h\), is used to update the parameters during the gradient descent. b Schematic representation of HGE in the frequency domain to extract the gradient from the spectral power density S(f), with the shading indicating the noise spectrum. Distinct frequencies f1, f2, f3, … with separation Δf are used to perturb multiple parameters in parallel. After mixing the output signal with a reference signal (here shown for f1), the derivative information is extracted from the total signal by removing undesired components at nonzero frequencies (i.e., noise and interference of other input perturbations) using a low-pass filter (blue shaded trapezoid). The horizontal dotted lines indicate the background white noise. c Effect of the magnitude of 1/f noise (expressed in terms of unfiltered noise at 1 Hz, Snoise (1 Hz) on the ability to extract the derivative using FD and HGE in simulation for the function h = 10w (error bars based on 1000 repetitions, α = 0.1, T = 1 s, w = 1 + δw). The variance of the HGE derivative is shown for various perturbation frequencies.
