Fig. 4: Simulation and energy consumption of an STPN network with multi-functional synapses.
a Sketch of the full STPN network. A frame of the Atari game is fed into two convolutional layers: Conv(kernel, stride) plus a ReLU activation function. The features are then fed into the m-STPN layer (blue ellipses and lines). The layer’s output is split into actions and a value by two fully connected linear layers. b, c Average reward as a function of agent steps during training for (b) three different implementations of the STPN layer and (c) five different ranges of Λ. Each curve represents the average reward of 16 agents with different random parameter initialization. The shaded area denotes the standard deviation. In the inset of c), the cases Λ = 0 and Λ = [0.08, 0.92] (i.e., the achievable device range) are shown. d Total synaptic weight (long- and short-term component) of a single synapse of the trained network (Smax{ΔF}) during an entire game. The Zoom-in additionally shows the long-term weight W in red and the ΔF as black bars. e Energy consumed by our memristors due to ΔF updates, i.e., voltage pulses with widths wp, fitted by a power-law. f Power consumed by our memristors due to different Decay bias voltages (Vbias). g Time evolution of the energy consumption of the synapse in (d) during an entire Pong game for a memristor (blue) and a pure GPU implementation (orange). Different energy contributions and the total energy are shown. h Histograms of all synapses in the network, indicating how many synapses consume a specific amount of energy during the whole Pong game. The two contributions ΔF and Decay are shown. For the Decay the worst case scenario: Vbias = 0.6 is assumed for all synapses. i Total energy histogram (ΔF plus Decay).
