Fig. 2: Eigenvalues of the SNN Jacobian for different models under standard deep learning initialization.
From: High-performance deep spiking neural networks with 0.3 spikes per neuron
a In α1-model the initial slope is \({A}_{i}^{(n)}={\alpha }_{i}^{(n)}=1\) and in the second regime the slopes of all neurons i are set to neuron-specific slopes derived from \({\alpha }_{i}^{(n)}=1\). Spikes are red, spiking threshold is purple, and three shades of green indicate evolution of \({V}_{i}^{(n)}\) for three different neurons i. b The eigenvalues of the α1-model Jacobian spread beyond the unit circle as \({B}_{i}^{(n)} \,\ne \, 1\), i.e., the network will experience exploding gradients at initialization. c In B1-model the initial slope is \({A}_{i}^{(n)}=0\) and in the second regime the slopes of all neurons i are set exactly to 1. d The eigenvalues of the B1-model Jacobian spread inside the unit circle as \({B}_{i}^{(n)}=1\), i.e., the network will not experience the exploding gradient at initialization.
