Fig. 5
From: Probabilistic metaplasticity for continual learning with memristors in spiking networks
Energy consumption and stability-plasticity tradeoff. (a) Energy consumption per sample and its breakdown during the parameter update phase for activity-dependent metaplasticity with gradient accumulation and probabilistic metaplasticity with individual and shared metaplasicity coefficients. (b) Distribution of the product of metaplasticity coefficient m and weight magnitude |w| in the hidden layer with probabilistic metaplasticity. We see that as more weights share m, lower fraction of weights assume low |mw| value. This indicates reduced plasticity in the network. (c) Mean accuracy across tasks vs. the energy consumption per sample for the split-MNIST task. We observe a tradeoff between continual learning performance and energy consumption as the metaplasticity coefficients are shared across weights.
