Fig. 4: Similarity between fMRI and VGG-16 representations.

a A schematic of how representational dissimilarity matrices (RDMs) were calculated. Images that all subjects viewed in the fMRI experiment were passed to the deep neural network (DNN) model (VGG-16). Then, the activation patterns of each DNN layer were extracted and pairwise distances (based on Pearson correlations) between images were calculated to form the neural network RDMs. Similarly, fMRI activation patterns were extracted for each of the same images, separately for each ROI and subject, to form the fMRI RDMs. Spearman correlations were then calculated between the neural network RDMs and the fMRI RDMs to quantify the correspondence (similarity) in representations. Photos taken by the corresponding author and the brain illustration from the Freepik website. b Spearman’s rank correlation coefficients between the neural network (VGG-16) RDMs and the fMRI RDMs. The neural network RDMs are separated by DNN layer, which represent different processing stages. Gray bars indicate the layers with significant RDM correlations between the neural network layer and fMRI ROI. Error bars denote the standard error.