Fig. 1: The problem of locality in spatio-temporal credit assignment.
a To illustrate the different learning algorithms, we consider three neurons within a larger recurrent network. The neuron indices are indicative of the distance from the output, with neuron i + 1 being itself an output neuron, and therefore having direct access to an output error ei+1. b Information needed by a deep synapse at time t to calculate an update \({\dot{w}}_{i-1,k}^{(t)}\). Orange: future-facing algorithms such as BPTT require the states \({r}_{n}^{({t}^{+})}\) of all future times t+ and all neurons n in the network and can therefore only be implemented in an offline fashion. These states are required to calculate future errors \({e}_{n}^{({t}^{+})}\), which are then propagated back in time into present errors \({e}_{n}^{(t)}\) and used for synaptic updates \({\dot{w}}_{i-1,k}^{(t)}\propto {e}_{i-1}^{(t)}{r}_{k}^{(t)}\). Purple: past-facing algorithms, such as RTRL, store past effects of all synapses \({w}_{jk}^{({t}^{-})}\) on all past states \({r}_{n}^{({t}^{-})}\) in an influence tensor \({M}_{n,j,k}^{({t}^{-})}\). This tensor can be updated online and used to perform weight updates \({\dot{w}}_{i-1,k}^{(t)}\propto {\sum}_{n}{e}_{n}^{(t)}{M}_{n,i-1,k}^{(t)}\). Note that all synapse updates need to have access to distant output errors. Furthermore, the update of each element in the influence tensor requires the knowledge of distant elements and is thus itself nonlocal in space. Green: GLE operates exclusively on present states \({r}_{n}^{(t)}\). It uses them to infer errors \({e}_{n}^{(t)}\) that approximate the future backpropagated errors of BPTT.
