Fig. 5
From: Brain-guided convolutional neural networks reveal task-specific representations in scene processing
(a) Illustration of our activation map analysis for an example image and network layer. We first took the absolute difference between two corresponding activation maps for any given stimulus and weighted those values by activation maps that would be expected by chance alone. We then took the dot product between the weighted difference maps and their corresponding task IU map (and the control map from the opposite task). See text for further detail. (b) Results from our activation map analysis described in the text (and illustrated in Fig. 5a). Error bars are 95% confidence intervals computed over 10 training runs, all Cohen’s d effect sizes were large (ranging between 0.68 and 1.36; corrected for small n). The network is considered to be using task relevant information when the reconstruction activation is higher when the reconstruction activation came from the same task as the IU map compared to when that same activation map was weighted by an IU map from the other task. For example, when tasked with classifying a given image when given neural data from the object tasks, the network used object relevant information more when compared to information that was more relevant to the function task.
