Fig. 4: Schematic of a hybrid CNN architecture for in materia speech recognition.

a, A schematic illustration of one core of the IBM HERMES project AIMC chip and its architecture containing 256 × 256 synaptic unit cells, each comprising four PCM devices organized in a differential configuration, ADC and/or digital-to-analogue converter (DAC) arrays and on-chip local digital processing units. b, Classifier architecture for the TI-46-Word dataset. A 64-channel RNPU preprocessing step converts audio signals into 64-D input to the AIMC chip with a downsampling rate of 10 (details in Extended Data Fig. 3). Batch normalization, activation functions and pooling operations are performed off-chip. c, Schematic representation of resource use for the three-layer CNN classifier in b for the TI-46-Word implemented on two tiles of the AIMC chip. d, CNN architecture for the GSC KWS task. After 64-channel RNPU preprocessing, a 6-layer CNN maps the inputs into 11 classes of known or 1 class of unknown targets. e, The mapping of CNN layers on the AIMC chip. In a fully pipelined implementation, 18 cores (out of 64) of the AIMC chip will be used. f, The confusion matrix of the GSC KWS task with HWA model training. Conv., convolution layer; FC, fully connected layer.