Figure 4
From: Downstream network transformations dissociate neural activity from causal functional contributions
Visual abstract of the MSA algorithm and the experiment. The experimental setup is depicted on the top. The ESN was trained on a time series prediction task in which a chaotic time series was fed into the network, and a readout layer was trained on the produced representations (deep blue) of the hidden layer. After the training, the teacher sequence was disconnected, and the network was expected to generate the future time steps of the time series (teal). Note that in ESNs, the hidden layer remains unaltered during the training session. The rest of the figure depicts the MSA algorithm that was employed to compute the causal contribution of each neuron in the hidden layer. MSA samples the space of all possible combinations of node groupings to estimate nodes’ causal contributions (red). To do so, it first permutes the players and expands each permutation configuration to dictate which combinations should be perturbed. In this schematic example, to acquire the outcome produced by BCA, the example player D was removed from the game. Then to have the outcome of BC, nodes D and A were perturbed. Therefore, MSA produces a multi-site perturbation dataset that contains the outcome produced by potentially tens of thousands of unique node groupings. To then isolate the contribution of individual nodes to each of these groupings, MSA contrasts two cases where the target node was perturbed and where it was not. The causal contribution of each node to the network's outcome is then the average of the node’s contributions to all groupings. Algorithm 1 (see “Multi-perturbation Shapley value analysis” in “Materials and methods”) describes the MSA computational pipeline.
