Fig. 1: Hebbian learning with memory.
From: Temporal Contrastive Learning through implicit non-equilibrium memory
A Consider a neural network with neuron i firing at rate ηi(t) and synaptic weight wij between neurons i, j. B In Hebbian learning, the weights wij are changed as a function g of the synaptic current sij(t) ~ ηi(t)ηj(t). Here, we generalize the Hebbian framework to a model in which weights wij are changed based on the history of sij(t), i.e., based on \({u}_{ij}(t)=\int_{-\infty }^{t}K(t-{t}^{{\prime} }){s}_{ij}({t}^{{\prime} })d{t}^{{\prime} }\) where K is a memory kernel. We find that non-monotonic kernels K that arise in non-equilibrium systems encode memory that naturally enables contrastive learning through local rules.
