Extended Data Fig. 5: Generative univariate RNC algorithm.

Generative univariate RNC generates stimulus images leading to aligned or disentangled in silico univariate fMRI responses for V1 and V4, while at the same time being as simple as possible. A batch of 1,000 random latent vectors is given as input to a GAN³⁷, which uses them to generate 1,000 images, and the PNG compression file size of these images is calculated. Next, these images are fed to the trained encoding models of V1 and V4, and the resulting in silico fMRI responses averaged across voxels, obtaining a one-dimensional univariate response vector of length 1,000, for each area. The univariate response vectors of the two areas are either summed or subtracted, based on the neural control condition univariate RNC is optimizing for, thus obtaining sum or difference scores. The latent vectors, PNG compression file sizes, univariate responses, and sum or difference scores are then fed to a genetic optimization algorithm^38,39, which uses them to create a new generation of latent vectors (by keeping the 250 best performing latent vectors, and recombining the remaining 750 latent vectors). At first the latent vectors are optimized using the sum or difference scores, so to result in images leading to in silico univariate fMRI responses for V1 and V4 closer to a threshold level. After this threshold is reached, the latent vectors are optimized using the PNG compression file sizes, so to result in images that are as simple as possible (while keeping the in silico univariate fMRI responses over the threshold). Finally, the new latent vectors are once again fed to the GAN, and the same steps are repeated over a new generation. After several genetic algorithm optimizations, this results in an image (that is, the best performing image from the last genetic optimization generation) that well controls neural responses following one of the four univariate RNC neural control conditions (that is, two alignment conditions where the in silico univariate fMRI responses of both areas are either driven or suppressed, and two disentanglement conditions where the in silico univariate fMRI response of one area is driven while the response of the other area is suppressed, and vice versa), while at the same time being as simple as possible. The images for the four neural control conditions are optimized independently of each other.