Fig. 2: Impact of injecting weight noise during training and inference on network accuracy.

a Test accuracy on CIFAR-10 obtained for different amounts of relative injected weight noise during training \({\eta }_{{\rm{tr}}}\) without inducing any perturbation during inference (\({\eta }_{\inf }=0\)). When \({\eta }_{{\rm{tr}}}\, > \, 8 \%\), the training convergence starts to become affected by the high noise and it is not possible anymore to reach the software baseline within the same number of epochs. The error bars represent the standard deviation over 10 training runs. b Test accuracy on CIFAR-10 obtained for the networks trained with different amounts of relative weight noise \({\eta }_{{\rm{tr}}}\) as a function of the weight noise injected during inference \({\eta }_{\inf }\). In most cases, \({\eta }_{{\rm{tr}}}\) can be increased above \({\eta }_{\inf }\) up to a certain point and still lead to comparable or slightly higher (within ≈0.1%) test accuracy than for \({\eta }_{{\rm{tr}}}={\eta }_{\inf }\). However, when \({\eta }_{{\rm{tr}}}\) becomes much higher than \({\eta }_{\inf }\), the test accuracy decreases due to the inability of the network to achieve baseline accuracy when \({\eta }_{{\rm{tr}}}\, > \, 8 \%\). Each data point represents the average over 10 training runs and 100 inference runs. c Test accuracy on CIFAR-10 as a function of \({\eta }_{{\rm{tr}}}={\eta }_{\inf }\). The error bars represent the standard deviation over 100 inference runs averaged over 10 training runs. d Top-1 accuracy on ImageNet as a function of \({\eta }_{{\rm{tr}}}={\eta }_{\inf }\). The error bars represent the standard deviation over 10 inference runs on a single training run.