Fig. 1: Overview of MICrONS dataset.

a, Depiction of functionally characterized volumes (left; GCaMP6s in green, vascular label in red) and EM data (right). Middle top, the overlap of the functional 2P (green) and structural EM (grey) volumes from which somas were recruited. Middle bottom, an example of matching structural features in the 2P and EM volumes, including a soma constellation (dashed white circles) and unique local vasculature (red arrows), used to build confidence in the manually assigned 2P–EM cell match (central white circle). All MICrONS data are from a single mouse. Scale bars, 5 μm. b, Deconvolved calcium traces from 100 imaged neurons. Alternating blue and white column overlay represents the duration of serial video trials, with sample frames of natural videos depicted below. Parametric stimuli (not pictured) were also shown for a shorter duration than natural videos. c, Schematic of the digital twin deep recurrent architecture. During training, video frames (left) are input into a shared convolutional deep recurrent core (orange and blue layers), resulting in a learned representation of local spatiotemporal stimulus features. Each neuron is associated with a location (spatial component) in the visual field (grey layer) to read out feature activations (shaded blue vectors), and the dot product with the neuron-specific learned feature weights (shaded lines, feature component) results in the predicted mean neural activation for that time point. CvT-LSTM, convolutional vision transformer, long short-term memory d, Top, depiction of 148 manually proofread mesh reconstructions (grey), including representative samples from layer 2/3 (red), layer 4 (blue), layer 5 (green) and layer 6 (gold). Bottom, presynaptic soma locations relative to visual area boundaries.