Extended Data Fig. 4: Neuro-stack experimental setup.
A simplified block diagram showing Neuro-stack setup including the (LEFT) Communication Layer, which uses a field-programmable gate array (FPGA, Xilinx Spartan 6 model) with a finite-state machine (FSM) built within it to allow for receipt of external messages (packets) via USB (for example, stimulation command) and streaming of the neural data. These packets are then processed and converted to serial peripheral interface (SPI) or Shift Register packets, and transmitted to the Neuro-stack integrated circuits (ICs, SPI: Stim and Sense IC; Shift Register: Spike and PLS IC). This FSM is bi-directional and thus also processes SPI packets received from the Neuro-stack and converts them into USB packets, which are then transmitted to the external Coral Development Board (Coral Dev Board) device. The FSM always begins with a Reset state after a reboot, and then enters an Idle state in which it waits for incoming packets. Once a packet is available, the FSM receives it byte by byte (Receive Byte) until the complete message is transferred (Receive Packet). The received packet is then being processed (Process Packet), converted into the appropriate interface (for example, USB to SPI), and transmitted to the Neuro-stack ICs (via SPI or Shift Register). Similarly, after the processing is done, the response packet from the ICs enters a state during which it can transmit the packet (Transmit Packet) byte by byte (Transmit Byte) externally. Once the transmission is done, the FSM goes back to the Idle state and waits for new packets unless the streaming of the neural data is taking place, in which case the FSM enters Process Packet state indefinitely until the recording is stopped. (CENTER) Coral Dev Board that can directly communicate with the Neuro-stack Communication Layer (via USB connection) using an Application Programming Interface (API, shown here) or a device (for example, Experimental Computer) with an installed GUI via USB (Fig. 1a). The Coral Dev Board contains a regular ARM-based central processing unit (NXP i.MX 8 M SoC) and a Google Edge Coprocessor, a tensor processing unit (TPU). The Neuro-stack API library runs on the ARM processor and contains a real-time pipeline for handling control and data flow to and from the Neuro-stack for each IC and Communication Layer (or FPGA). The Input Queue handles streams of both neural data and acknowledgment receipts from the Communication Layer and redirects them to the appropriate block on the Coral Dev Board responsible for each ICs (for example, Sense, Stim,… Process). The Neuro-stack Control block contains all of the API functions, which are then multiplexed to additional layers responsible for wireless (via Server Interface) or wired (via Local Interface) communication with the Experimental Computer. Additionally, the Neuro-stack Control block also contained functions for controlling the TPU, such as loading/saving the machine learning model (TensorFlow Lite Model) to/from the Memory block, redirecting the data streams directly towards the TPU, and receiving the TPU’s output once it is ready. The incoming neural data streams can also be stored locally in Log Memory or transferred to external storage on the Experimental Computer through the Neuro-stack Control block. Furthermore, an LED light can be triggered (to turn on/off) through available general-purpose input/output (GPIO) pins for synchronization purposes. These triggered on/off events are internally temporally aligned with the incoming neural data in order to synchronize it with data from eye-tracking cameras. (RIGHT) A local network can be created either by using a separate access point (shown here, for example, router, hotspot, etc.), or by the Coral Dev Board, which contains a network controller that can support access point topology and thus can create its own local network. This wireless mode means that a server is created on the Coral Dev Board to allow for other devices, such as the Stimulus Presentation device (for example, iPad used to present the verbal memory task) or Experimental Computer (for example, to view neural data in real-time) to access the Neuro-stack API functions. Security warning points to the importance of a safe wireless connection, implemented by X.509 certificate authentication. Wired mode is also supported through Local Interface block (for example, Experimental Computer connected via USB-C). All devices connected to the local network use Network Time Protocol (NTP) to log events with timestamps fetched from a common server in order to synchronize them. (BOTTOM) The structure of the USB packets sent from the Coral Dev Board to the Neuro-stack Communication Layer, which contains up to 520 bytes that describe the type of Command, Board ID to address specific analog layer, Spike byte, phase-locked stimulation (PLS) byte, and Payload for additional information where its length (Payload Length) depends on the type of command. The packet also contains bytes for error codes (Error) and a cyclic redundancy check (CRC) to detect accidental changes in the raw packets. The communication layer extracts a relevant portion of the USB packet, converts it to desired interface (SPI or Shift Register), and transmits it to the addressed IC.
