Fig. 3: Explaining 25D feature maps using interpretable visual features.

a The 25 feature maps in Fig. 2 were merged into a 25D feature map, where each location’s preferred feature was represented by a 25D vector. b 21 visual features were projected into the 25D object space (blue) and compared with the neural features (red). c Similarity between 25D neural features of three monkeys and 21 features (see Methods). The squared cosine angle between two 25D feature vectors was used to quantify their similarity (inset). Box plots show the median (line), quartiles (boxes), range (whiskers), and outliers (circles); n = 3321 brain locations from three monkeys. d–f Feature selection. A set of most explanatory features were selected by adding one feature at a time to optimize the total explained proportion of neural responses at each step (see Methods). d The procedure for quantifying the explanatory power of a set of features (see Methods). f₁ and f₂ indicate the 25D preferred features estimated using two halves of the data; e₁ and e₂ indicate the selected features. e Two baseline models. 1) Principal components of the responses of AlexNet units in four layers (fc6 and fc7, before and after ReLU) to 200k ImageNet images used to construct the 25D object space. 2) WordNet labels. Each selected label was required to contain no less than 100 and no more than 2000 out of all 2500 representative images (see Fig. S5c). f Goodness-of-fit, normalized by the noise ceiling (see Methods), is plotted against the number of features for multiple models. The model in (c) is denoted by the black line, with the arrow indicating the optimal feature number ( = 7). Additional features were derived by first orthogonalizing the neural features with respect to the selected seven features, and then performing PCA on the orthogonalized features. Incorporating these orthogonal features continued to increase the goodness-of-fit (purple line), until 9 features were selected (purple arrow). Orange lines indicate PCs of AlexNet units, with different symbols representing different layers. The blue line indicates the WordNet labels, which underwent the same feature selection process as the model in (c). Source data are provided as a Source Data file.