Fig. 3: Dynamics of the modulation factor.
From: Introducing principles of synaptic integration in the optimization of deep neural networks
a Mean and standard deviation dynamics of the modulation factors for a 10 × 256 tanh network, with 10% dropout, trained with BP, SGD, and GRAPES modulation for 200 training epochs. The modulation factors are recorded for each layer every 10 epochs. b Distribution of the modulation factors at the end of training.
