Fig. 3: Motion recognition and prediction.

a Programmed light pulses representing object motion are used as input to the retinomorphic PMA. The motion consists of three frames (illumination time of 50 ms per frame), and it is played at three speeds; slow (3 s for 3 frames), medium (1.5 s), and fast (0.6 s). Only the photomemristor output currents recorded after playing the last frame (h₃) are used as features for recognition by the readout network. b Feature vectors of hidden memory states (h₃) of the three motions (slow, medium, fast) obtained after playing the last frame with V_bias = 1.0 V. The last frame is recognized for all speeds. Moreover, h₃ contains hidden states representing imprints of previous frames and the speed at which they were played. c Imprint of previous object positions for different speeds of motion. The imprint factors are calculated by averaging the normalized memory of pixels #9,10,11,13,16,17,18,19,20,22,24,25 (same indices as in Fig. 2b) in b. The selected pixels are all illuminated by optical pulses during object motion. High-speed results in a stronger imprint because of the rate dependence of hidden memory states. The error bars represent the standard deviation. d Training and validation accuracy of the readout network. e Confusion matrix of motion speed recognition, showing 100% test accuracy. f Structure of the RP-RC system for motion prediction. X₁ indicates the first frame for prediction, AE is the autoencoder network, and Y_i is the predicted output frame. g, h Prediction of an object moving to the left (g) and moving to the right (h) with input X₁. The predicted output frame of the autoencoder, h, contains imprints of previous frames as hidden states. The final output Y is obtained by applying a step function to h. i Predicted position and moved distance (D) at t = 9 s following the detection of an object moving to the right at three different speeds.