Fig. 1: Plasticity loss in Continual ImageNet.
a–c, In a sequence of binary classification tasks using ImageNet pictures (a), the conventional backpropagation algorithm loses plasticity at all step sizes (b), whereas the continual backpropagation, L2 regularization and Shrink and Perturb algorithms maintain plasticity, apparently indefinitely (c). All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.
