Fig. 3: Image reconstruction from population activity.
a, b Schematic of the image reconstruction models. a In the all-cell model, each feature value was reconstructed by all cells. In the cell-selection model, the feature value was reconstructed by selected cells for each feature. The cell selection was based on the response prediction model for each cell; each cell participates in the reconstruction of features that the cell encodes in the response prediction model. b Details of the image reconstruction model. For each Gabor feature, j, feature values (Fji, i: trial number across stimuli and trials, j: Gabor feature number) were independently regressed (weights: Hjk, k: cell number) by multiple cell responses (Rki) to the image (Ii) in the ith trial. Then, a set of reconstructed features (F1i, F2i, …, F1248i) was transformed into an image \(({\hat{\mathrm{I}}}_i)\). The flow of the reconstruction model is represented by black arrows from the bottom to the top. c Examples of reconstructed images from the main datasets (dataset 1; 200 images). Stimulus images presented during imaging (top), images that were reconstructed using the all-cell model (all cell, middle) and using the cell-selection model (cell selection, bottom) are shown. Each reconstructed image was averaged across trials. The reconstruction performances (R and CD) were computed for each trial, and trial-averaged performances are presented below each reconstructed image. d Examples of reconstructed images from other datasets (dataset 2; 1000–2000 images). e Distributions of R (top) and CD values (bottom) for the all-cell model (black lines) and the cell-selection model (red lines) in the example plane shown in Figs. 1 and 2 (n = 200 images reconstructed using 726 cells from a plane). Vertical lines indicate median values. f, g R f and CD g of dataset 1 (black lines and bars) and of dataset 2 (green lines) across planes. *p = 0.006 in f and p = 1.8 × 10−5 in g using the signed-rank test (n = 24 planes for dataset 1). Some of the stimulus images in b–d are adapted from the databases in refs. 56,57,58 with permission. Source data are provided as a Source Data file.
