Fig. 3: Neuromorphic overparameterisation.
From: Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks
Train and test MSE for selected architectures when predicting future values of the Mackey-Glass equation when varying the number of output parameters P (i.e., FMR output channels) for training set lengths of a 50 and c 100. MSE is the mean of 50 random combinations of outputs across 6 different prediction tasks (Mackey-Glass, t+1, 3, 5, 7, 9, 11). Shading is the standard error of the MSE over 50 trials of randomly selecting features. b, d Improvement factors (min(MSEUP) / min(MSEOP)) when moving from underparameterised (UP) regime to overparameterised (OP) regime. Each network shows three regimes: an underparameterised regime when the number of parameters is small, an overfitting regime where the number of parameters is near the size of the training set and an overparameterised regime when the number of parameters exceeds the size of the training set. Example train and test predictions for 3 series architecture in the underparameterised regime (e), overfitting regime (f) and overparameterised regime (g).
