Fig. 3: fMRI identification accuracy patterns using brain-aligned semantic vectors across visual cortex regions and alignment parameters.
From: Brain-aligning of semantic vectors improves neural decoding of visual stimuli
Heatmaps showing zero-shot identification accuracy for visual stimulus decoding using brain-aligned semantic vectors derived from different regions of interest (ROIs) in the visual cortex. The results are shown for a CLIP-based and b GloVe-based semantic vectors under both perception and imagery conditions. The rows represent different visual cortex ROIs (V1–V4: primary and secondary visual areas; LOC lateral occipital complex, FFA fusiform face area, PPA parahippocampal place area). The columns represent different brain-alignment parameters (α), where “Original” indicates unaligned pretrained vectors, α = 1 indicates reconstruction without brain alignment, and decreasing α values (0.1–0.0001) indicate increasing degrees of brain alignment. The color intensity reflects identification accuracy, with warmer colors indicating better performance. The optimal brain-alignment parameters varied by semantic vector type and brain region, with V4-derived brain-aligned vectors showing particularly strong performance for both CLIP (α = 0.1) and GloVe (α = 0.01) across perception and imagery conditions. n = 5 subjects from the GOD dataset.
