Fig. 6: Consistency of identification accuracy improvement between the optimal CLIP-based brain-aligned vectors and original CLIP-based vectors across all three modalities.

a Category-specific accuracy improvements following brain alignment of CLIP vectors across the fMRI (blue circles), MEG (red squares), and ECoG (green triangles) modalities. The brain-aligned vectors were trained on the fMRI V4 region data, with the optimal α parameters determined from the fMRI V4 region identification analysis and applied across all the modalities. The categories are grouped by semantic clusters (C0–C8) derived from whole ImageNet clustering analysis (Supplementary Fig. 6). Each modality tested the same n = 50 categories. The y-axis shows the accuracy improvement relative to the original CLIP vectors. The dotted vertical lines separate the cluster boundaries. The consistent patterns in identification accuracy improvements across modalities demonstrate robust cross-modal generalization of V4 region brain-alignment benefits. b Pearson correlation matrices comparing the accuracy patterns across neuroimaging modalities for original performance (left), improvement (center), and final performance (right). All the results use brain-aligned vectors trained on fMRI V4 region data with optimal α parameters derived from fMRI V4 region identification analysis. The values indicate correlation coefficients between modality pairs. The high correlations in the accuracy improvement matrix (center) demonstrate that categories benefiting from V4 brain alignment in fMRI consistently benefit across the MEG and ECoG modalities, supporting the robustness and generalizability of the brain alignment approach.