Fig. 3: Step-by-step description of the main signals ruling the autonomous navigation.
From: A self-adaptive hardware with resistive switching synapses for experience-based neurocomputing
a The FPGA records the current position of the agent and b triggers the gate voltage signal of the synaptic devices to start the integration phase of the nearest neurons. Once a neuron fires, all the integration signals are discharged by switching on a transistor in parallel to the capacitor used for integration. After the fire event, the corresponding synaptic connection is brought high (c) and the current position of the agent is updated (d); the threshold of the new internal state rises as a consequence of the internal state partial set (e); the procedure (a–e) is repeated at every movement of the agent. f If a position (i, j) is accessed consequent times, it plastically adapts the corresponding internal thresholds causing a gradual increase of the threshold VTH; the neuronal threshold plastic adaptation is also used to map the penalties, by increasing the corresponding VTH (g), and the rewards, by decreasing the corresponding VTH (h). Note that the gradual increase of the neuronal threshold is bounded to the effective multilevel capability of the RRAM devices (i). During the ordinary movement, the synaptic connections from one position to another are potentiated or depressed for the STDP mechanism (j), while, on the other hand, the penalty positions always undergo depression, due to reinforcement learning (k). Note that the synaptic connections are always potentiated if the agent does not come back. If rewarded positions run into a penalty due to the dynamic evolution of the environment, the corresponding internal thresholds rise slower than the ordinary positions, due to the firing history and the different fire excitability (l).
