Fig. 6: Evaluating pipelines across all criteria.

Each data-point represents one pipeline (n = 768), with colour and shape reflecting which criteria are met. Criterion (I): Avoiding spurious differences (“PDiv ranking”). We consider pipelines as optimal if they are in the top 20% in terms of the global rank based on PDiv (Portrait Divergence) calculated as the average rank achieved in each dataset. We further show pipelines which fulfil all other criteria while being among the top 50% in terms of the average global rank. The maximum average PDiv among the top 50% pipelines is 0.169. Criterion (II): Detecting true experimental differences (“propofol”). Suitable pipelines should detect a significant effect for propofol, in the right direction, in both propofol datasets, i.e. a pipeline is excluded if it fails to detect the expected effect in either of the two propofol datasets. The Y axis reports the maximum between the two t-statistics obtained for the two propofol datasets, so pipelines satisfy the sensitivity criterion if they score < 1.96 on this axis (i.e., find a significant effect for propofol, in the right direction, in both propofol datasets). Criterion (III): Detecting inter-individual differences (“within-between”). A pipeline fails this criterion if the resulting networks are more similar between than within subjects more than 50% of the times, for any of the four test–retest datasets. Criterion (IV): avoiding motion-induced differences (“motion”). A pipeline fails this criterion if its PDiv has a significant correlation with differences in head motion in any of the four test–retest datasets. Criterion (V): non-empty networks. As a final check, we also exclude any pipelines that remove all connections from a network, in any of the four test–retest datasets. “Fail both” refers to pipelines failing in terms of motion and within-between criteria, while “Pass both” refers to pipelines which satisfy both of these criteria. Points circled in purple represent pipelines that produced empty networks. Overall, 9 pipelines satisfy all criteria in all datasets; this number grows to 35 if a more liberal PDiv criterion is adopted (top 50% global rank). Source data are provided as a Source Data file.