A network correspondence toolbox for quantitative evaluation of novel neuroimaging results

Abstract

The brain can be decomposed into large-scale functional networks, but the specific spatial topographies of these networks and the names used to describe them vary across studies. Such discordance has hampered interpretation and convergence of research findings across the field. We have developed the Network Correspondence Toolbox (NCT) to permit researchers to examine and report spatial correspondence between their novel neuroimaging results and multiple widely used functional brain atlases. We provide several exemplar demonstrations to illustrate how researchers can use the NCT to report their own findings. The NCT provides a convenient means for computing Dice coefficients with spin test permutations to determine the magnitude and statistical significance of correspondence among user-defined maps and existing atlas labels. The adoption of the NCT will make it easier for network neuroscience researchers to report their findings in a standardized manner, thus aiding reproducibility and facilitating comparisons between studies to produce interdisciplinary insights.

Introduction

Standardized scientific nomenclature can facilitate the effective communication of concepts related to complex biological systems. Neuroscience has a long history of classifying distinct anatomical regions of the brain across multiple spatial scales, from the cortical lobes, discrete gyri and sulci of the cerebral cortex and subcortical nuclei, to Brodmann’s cytoarchitectonic areas¹. Attempts have also been made to categorize well-defined neural circuits and systems based on various anatomical and functional criteria, such as the cortico-spinal tract, cortico-striato-thalamic circuits, and the dorsal and ventral visual processing streams². Such attempts at standardization facilitate communication about brain organization between researchers and assist in educating students.

Modern neuroimaging methods, particularly those relying on magnetic resonance imaging, have revolutionized our capacity to non-invasively map different aspects of brain structure and function in living humans. A key development has been the ability to map functional brain networks, or groups of brain areas that exhibit functional synchrony. As a result of the growing popularity of this approach, many findings from neuroimaging studies are described in terms of the brain networks, rather than the brain areas, where they are localized. However, this is often done in an ad hoc way due to the lack of standardization in the naming of these networks. This inconsistency in scientific reporting complicates comparisons of findings across studies and limits the integration of novel discoveries³.

The emergence of functional brain network atlases, which are now routinely used in cognitive and network neuroscience⁴, should, in principle, contribute to standardization in the naming of functional networks. These atlases typically assign parcellated cortical and/or subcortical regions to one of a set of large-scale brain networks with varying topographies, ranging from largely contiguous unimodal territories (e.g., visual network in occipital cortex) to associative systems that span multiple spatially segregated and distributed regions across the brain (e.g., default network encompassing frontal, temporal and parietal regions). The atlases also vary in how they name large-scale brain networks and in terms of the number of brain networks, as well as their approach to defining the networks, characterizing their spatial topography, and deciding how they should be labeled (e.g., based on their anatomical location or purported cognitive correlates)⁵. These differences between atlases lead to further variability and uncertainty in scientific reporting.

Recognizing these challenges, the Organization for Human Brain Mapping (OHBM) established a Best Practices committee on large-scale brain network nomenclature, following the success of earlier similar consensus-building initiatives aiming to enhance standardization by providing reporting guidelines for the field^6,7. This committee, formed in 2020, is known as the Workgroup for HArmonized Taxonomy of NETworks (WHATNET). Initial attempts by the workgroup to conceptually identify convergence across large-scale functional brain networks led to discussion and enumeration of the multiple challenges inherent to this enterprise⁸. The workgroup surveyed hundreds of scientists and solicited putative network names based upon renderings of various network spatial topographies. Results from the survey, described in detail in the initial report from WHATNET⁸ and publicly available (https://osf.io/3fzta/), indicated that networks based in non-distributed unimodal brain regions (somatomotor and visual systems) could be readily and reliably identified. The somatomotor network showed 97% agreement amongst raters, while the visual network showed 92% agreement amongst raters. Only one spatially distributed network, the default network, demonstrated consensus agreement in name and topography, with 93% agreement amongst raters. Other networks, such as those sometimes referred to as salience or frontoparietal, showed less consistency. These findings indicate that consensus agreement in nomenclature is limited to the extreme poles of the canonical cortical hierarchy⁹, anchored at one end by the visual and somatomotor systems and on the other by the default network. In between these systems is an expanse of cortex with observable network structure, but with little agreement between researchers regarding their spatial topography or nomenclature. Such variability poses a major problem for any attempts to compare or integrate findings across studies.

In response to the WHATNET survey findings, here we present the Network Correspondence Toolbox (NCT). This toolbox provides researchers with a practical solution for labeling novel observed fMRI activation, functional connectivity patterns, or other thresholded neuroimaging maps based on a quantitative evaluation against multiple currently published atlases simultaneously. The NCT also allows users to assess the spatial correspondence between networks in new and existing atlases. We demonstrate the utility of the NCT in providing network labels to exemplar neuroimaging maps based on a quantitative evaluation of spatial correspondence with multiple published network atlases. Quantitative evidence of correspondence is determined by the magnitude of the Dice coefficient, in addition to spin test permutations for robust statistical assessment of significance. We conclude with a recommendation that future network and cognitive neuroscience reports include an evaluation of the correspondence between their network labeling schemes and multiple published network atlases produced by independent research laboratories, as facilitated by the NCT. This approach transparently acknowledges and quantitatively addresses the ambiguity inherent in assigning labels to topographic brain maps and encourages greater alignment across network neuroscience studies by objectively assessing the convergence or divergence between new findings and published network labels and schema. Importantly, the NCT is not prescriptive regarding the largely intractable problem of inconsistent network nomenclature. Rather, it is a tool to facilitate comparison and interpretation to be drawn across atlases, ultimately resulting in greater standardization in reporting of results.

We begin by illustrating the network correspondence problem by computing network overlap across sixteen widely used functional brain atlases. Next, as another quantitative example, we demonstrate the correspondence between two widely-used atlases that delineate the cortex into 17 discrete networks: Yeo2011-17 and Gordon2017-17. These examples serve to illustrate the network nomenclature problem and to motivate NCT usage. Then, we provide step-by-step guidelines for NCT usage. Finally, we report several examples of how the NCT can be used by treating publicly available Human Connectome Project (HCP) task fMRI results and UK Biobank independent component analysis (ICA) maps as “novel” findings for demonstration purposes. These specific maps were chosen to illustrate how the NCT performs in cases where the input data correspond with existing networks to a strong degree, as well as in more challenging scenarios where correspondence is lower.

Results

Network correspondence toolbox

Anatomical localization of patterns of brain activity is tedious¹⁰, but many digital and paper atlases are available to facilitate the process¹¹. Localization to large-scale functional brain networks is a more recent development, with standardized tools in demand (see¹² for a recent introduction of probabilistic network localization). The NCT flexibly permits the assessment of spatial correspondence between several target reference atlases and novel brain maps including, but not limited to, functional connectivity and task activation results, as well as other maps in standard space¹³. Note that some of the included atlases included in the NCT contain multiple versions: Yeo2011, Schaefer2018, Kong2021, Power2011, Shen2013, Gordon2017, Cole-Anticevic2019, ICA-UK Biobank, ICA-HCP, ICA-BrainMap, and Shirer2012^{14,15,16,17,18,19,20,21,22,23,24,25} (Fig. 1 and Supplementary Table 1). The most recent release of the NCT (v 0.3.1) includes 23 brain atlases. A full list of currently available atlases in the NCT can be found in https://pypi.org/project/cbig_network_correspondence.

Fig. 1: Example atlases included in the Network Correspondence Toolbox (NCT).

The NCT is a toolbox that facilitates exploration of network correspondences across multiple functional network atlases as well as quantitative comparison of novel neuroimaging results with multiple atlases. Ten atlases are shown here for illustration purposes. In this example, the Yeo 17-network atlas in fsaverage6 space (center) serves as the reference atlas. All other surrounding atlases in different spaces are projected to the fsaverage6 space to compute Dice overlap coefficients with the reference networks. A full list of currently available atlases in the NCT can be found in https://pypi.org/project/cbig_network_correspondence.

Network correspondence: understanding the problem across multiple atlases

We considered 16 widely used functional brain atlases including the Yeo2011, Schaefer2018, Kong2021, Power2011, Shen2013, Gordon2017, Cole-Anticevic2019, ICA-UK Biobank, ICA-HCP, ICA-BrainMap, and Shirer2012^{14,15,16,17,18,19,20,21,22,23,24,25} (Fig. 1 and Supplementary Table 1). To compute the network overlap across atlases, we treated each atlas as the reference atlas and projected other atlases into that reference atlas. Between each reference network and each network from all other atlases, we computed the Dice coefficient. This allowed us to compare the voxel-wise agreement between the reference network and all other networks, where 0 is no correspondence and 1 is total correspondence. Assuming there are K networks in the reference atlas and M networks in another atlas, we generated a KxM network overlap matrix, where the k-th row and m-th column represents the Dice overlap coefficient between network k of the reference atlas and network m from the other atlas. A high Dice coefficient indicates high spatial overlap between the two networks from different atlases.

Dice coefficients between all 16 atlases were computed and examined. The non-symmetrical matrix shows relative differences due to atlases referencing in the upper and lower triangles (Fig. 2). Each matrix sub-block reveals patterns of correspondence (with total correspondence within-atlas on the diagonal). Non-square sub-blocks of the matrix are due to differences in the number of networks identified between atlases, which ranged from 7 (Yeo2011) to 25 (Smith2013; Miller2016). Distributed spatial topographies of functional networks across atlases show some reliable correspondence, based on the observable sub-block diagonal structure.

Fig. 2: Hierarchical structures of the network similarity matrix (Dice overlap).

The nested stochastic block model with 15 clusters was applied onto the network similarity matrix where Dice coefficients were calculated between each pair of networks from different atlases. The network similarity matrix is a 230 × 230 matrix since there are 16 atlases with 230 networks. Each block corresponds to a network cluster. Representative network names within each cluster are highlighted here. See Supplementary Fig. 1 for clustering results with atlas associations. We emphasize that the hierarchical clustering is not meant to generate a new consensus atlas; rather it is a way for us to examine convergence and divergence across atlases. Prim primary, med medial, lat lateral, vis visual, assoc association, par parietal, occ occipital, ant anterior, MTL medial temporal lobe, front frontal, ven ventral, attn attention, tem temporal, SM somatomotor, post posterior, SAL salience, subcor subcortical, cing cingulo, operc opercular.

Among the 16 atlases included, differences in numbers of networks included some cases of non-independent solutions that were derived from the same dataset or from the same laboratory. For this reason, the present investigation cannot determine the optimal solution for the number of large-scale networks in the human brain, as assessed using fMRI. Based on the Dice overlap, hierarchical structures of the network similarity matrix were examined using a nested stochastic block model²⁶, with 15 clusters applied (the average number of networks across atlases). It is important to note that this clustering is not meant to generate consensus networks across atlases. Rather, the clustering algorithm permits an examination of the spatial correspondence across atlases, indexed by atlas source, and specific network names. The network probability maps of each cluster are displayed in Fig. 3.

Fig. 3: Spatial topography of the 15 network clusters.

See Supplementary Fig. 1 for atlas contributions and hierarchical relations. Warmer colors indicate greater concordance across atlases. We emphasize that these maps are not meant to be consensus networks, i.e., we are not proposing a new atlas. Instead, these maps are meant to illustrate the convergences and divergences between existing atlases.

We visually inspected each cluster and associated network names for parcellations that provided network labels. The spatial extent of the network probability maps and their corresponding labels (ignoring subnetwork designations) were similar across atlases for many clusters. All clusters were bilateral. Cluster 5, comprising medial prefrontal and medial parietal cortex, inferior parietal lobule, lateral temporal and medial temporal lobe, inferior frontal gyrus, and superior middle frontal gyrus, was nearly uniform in being labeled the “default network” across atlases. However, some functional ensembles were diversely named, and distinct inconsistencies were noted.

The distributed network of Cluster 12, including lateral prefrontal cortex, anterior inferior parietal lobule, posterior inferior temporal gyrus, and dorsal anterior cingulate cortex, also showed strong correspondence across multiple atlases, but the atlases used diverse labels, such as “frontoparietal”, “control”, and “central executive”. This network is well known and identified for its role in the executive control of information processing in the human brain²⁷. The divergent names largely point to the same cognitive construct even in the absence of consistent and consensus nomenclature across atlases²⁸.

Convergence was observed for perceptual and motor networks. Probability maps of Clusters 2 and 3 converged in the occipital lobe, with consistent labels related to vision (“visual”). Rostral to the occipital networks was Cluster 4, which extended dorsally into superior parietal cortex and ventrally into inferior temporal cortex This network was variously labeled “dorsal attention”, “visual association”, and “visual”. Cluster 1 included two regions rostral to those identified in Cluster 4, the superior parietal lobule and middle temporal motion complex, as well as the putative frontal eye fields and inferior precentral sulcus. These distributed regions were predominantly labeled “dorsal attention” in addition to “visuospatial” and “premotor”.

Motor and somatosensory cortex, anterior and posterior to the central sulcus, include the precentral and postcentral gyri, and extend medially into the juxtapositional lobule (supplementary motor area). Cluster 10 included these areas with reliable labels of “somatomotor” or “motor”. The dorsal and medial extent were specifically recapitulated in Cluster 8, with labels of “somatomotor”, and specifically “hand”, “foot”, and “dorsal somatomotor”. Cluster 11 included the ventral extent of motor and somatosensory cortex, descending through the middle insula, and terminating in the superior temporal gyrus. This cluster was split in names between “somatomotor” and “auditory” labels across different atlases.

Finally, convergence of the “ventral attention”, “salience”, and “cingulo-opercular” networks was apparent in Cluster 9, with regions including anterior insula, middle and dorsal anterior cingulate cortex, and lateral parietal cortex. For example, the salience/ventral attention network-A from the Yeo2011-17 atlas largely overlapped with the cingulo-opercular network from the Gordon2017 atlas. However, the salience network label has also been applied to anatomically different regions as well, with poor convergence between some atlases. The salience network as labeled in the Gordon2017 17-network, Yeo2011 7-network, Shirer2012 14-network, and Shen2013 8-network show distinct distributed spatial topographies (Fig. 4).

Fig. 4: Networks with similar names can show different spatial topographies.

“Salience” networks from four different atlases. These networks are labeled using similar nomenclature across multiple atlases, though they span different, largely non-overlapping anatomical locations.

Five network probability maps were more ambiguous in their topography and naming. Clusters 13–15, and 6 included two to three discrete regions in each hemisphere. These regions may be part of larger named systems but can present as unique ensembles, depending on the atlas used. Cluster 13, including orbitofrontal and anterior temporal regions, was labeled “limbic” and “anterior medial temporal” (note these regions are often left unlabeled in some atlases due to heavy overlap with susceptibility artifact in fMRI). Cluster 14 included ventral retrosplenial cortex, inferior temporal and lateral occipital regions and was diversely labeled “context”, “parietal-occipital”, and “default”. Cluster 15 included dorsal retrosplenial cortex and dorsal posterior precuneus, and was labeled “control”, “salience”, and “medial parietal memory”. Cluster 6 included the inferior frontal gyrus and superior lateral temporal regions, and was labeled “TempPar”, “language”, and “ventral attention”. The topography of Cluster 6 resembles task-driven language function^24,29,30. Problematically, the cognitive network terms associated with Clusters 14, 15, and 6 are incongruent.

Cluster 7 was unique in that it only included networks identified from atlases derived using ICA. These networks were given a numerical ID rather than labels derived from anatomical or cognitive features and originated from HCP and UKBiobank. Brain regions included dorsomedial prefrontal cortex, anterior insula, inferior parietal lobule, lateral temporal cortex, dorsal anterior precuneus, and posterior superior frontal gyrus.”

Network correspondence: understanding the problem between two specific atlases

As another quantitative demonstration of the network correspondence problem that motivated the development of the NCT, we evaluated network-to-network spatial correspondence between two widely used atlases. For this illustration, all 136 network pairs from the Yeo2011-17 and Gordon2017-17 atlases were examined (Fig. 5; See Supplementary Table 2 for Dice coefficients and corresponding p-values). Overall, we observed good correspondence between several of the networks across these two atlases. For example, Gordon2017-17 Lateral Visual and Medial Visual networks significantly corresponded to the Yeo2011-17 Visual A and B networks, respectively. Specific Hand, Face, and Foot Somatomotor networks in the Gordon2017-17 showed significant correspondence with Somatomotor networks A and B in Yeo2011-17. However, notable discrepancies also emerged, particularly with regard to network name and purported function. For example, the Gordon2017-17 Auditory network also spatially corresponded to the Yeo2011-17 Somatomotor network B. While the Gordon2017-17 Dorsal Attention network showed significant correspondence with the Yeo2011-17 Dorsal Attention networks A and B, the Gordon2017-17 Premotor network also significantly corresponded with the Yeo2011-17 Dorsal Attention network B. The Gordon2017-17 Cingulo-opercular network significantly corresponded with the Yeo2011-17 Salience / Ventral Attention networks A and B. Yet a distinct Gordon2017-17 Salience network also corresponded with Yeo2011-17 Salience / Ventral Attention network B. The Gordon2017-17 Anterior Medial Temporal Lobe network significantly corresponded to Yeo2011-17 Limbic A network (which comprises the anterior temporal lobes). In robust agreement, the Gordon2017-17 Frontoparietal network significantly corresponded to the Yeo2011-17 Frontoparietal Control networks A and B. However, the Gordon2017-17 Parietal Memory network significantly corresponded to the Yeo2011-17 Frontoparietal Control network C. In another example of seemingly good spatial alignment, the Gordon2017-17 Default network significantly corresponded to the Yeo2011-17 Default networks A and B. However, the Gordon2017-17 Default network also showed significant correspondence with Yeo2011-17 Frontoparietal Control network B, highlighting the importance of considering the relative magnitude of the Dice coefficient. The Gordon2017-17 Language network significantly corresponded with the Yeo2011-17 Temporal Parietal network, as well as Default network B. The Gordon2017-17 Context network and Posterior Medial Temporal Lobe network both significantly corresponded to the Yeo2011-17 Default network C. Notably, the Yeo2011-17 Limbic B network (which comprises orbitofrontal cortex) showed no correspondence with any Gordon2017-17 networks.

Fig. 5: Illustration of NCT usage to explore network correspondence between two atlases.

In this example, we explore overlap between the Yeo2011 17-network atlas (reference atlas) and the Gordon2017 17-network atlas. A The user specifies the name of the reference atlas (here, Yeo2011-17) and the name of comparison atlas (here, Gordon2017-17). The Yeo 17-network atlas is in fsaverage6 space, while the Gordon2017 17-network atlas is in fs_LR_32k. In this case, the reference atlas space is fsaverage6. Therefore, the NCT projects the Gordon2017 17-network to fsaverage6 space. B The NCT computes the Dice overlap between networks from Yeo 17-network atlas and Gordon2017 17-network atlas. The NCT also performs a spin test to test whether networks significantly overlap. C The NCT reports the network correspondence between these two atlases using an overlap heatmap plot, where the k-th row and m-th column represents the Dice overlap coefficient between network k of the Yeo 17-network atlas and network m from the Gordon2017 17-network atlas. A high Dice coefficient indicates high overlap between the two networks. Brighter colors indicate higher overlap, darker colors indicate lower overlap. The “*” indicates two networks significantly overlap (p < 0.05). Most Yeo2011 networks overlap with at least one network in the Gordon2017 atlas. The NCT also provides a summary table showing the exact Dice coefficients and p-values (see Supplementary Table 2). The NCT uses network names from the original paper for each atlas. Dors dorsal, attn attention, sal salience, ven ventral, temp temporal, par parietal, lat lateral, vis visual, med medial, SM somatomotor, cing cingulo, operc opercular, ant anterior, MTL medial temporal lobe.

This Yeo2011-17 and Gordon2017-17 example further illustrates the problems of (1) incongruence in network nomenclature, and (2) varying levels of network correspondence across atlases. A pair-wise spatial correspondence analysis of all networks included within all sixteen atlases is outside the scope of the present report. However, the NCT retains this functionality for interested readers. In-depth pairwise assessment between brain network atlases, in addition to benchmarking of novel neuroimaging results against networks defined across multiple atlases, is a core function of the NCT. Due to spatial and nomenclature variability across atlases, the OHBM Best Practices committee now recommends that users use multiple atlases originating from different research laboratories for reporting correspondence between novel results and existing network labels. The recommendation to report results in reference to multiple atlases comes from the observation that even networks with similar nomenclature across atlases (e.g., “salience”) may not share the same spatial topography (Fig. 4).

How to use the NCT

For each network of a reference atlas, the magnitude of spatial correspondence with an empirical brain map, and the statistical significance of this correspondence, can be determined using the NCT. All atlases and fMRI results are projected from a standard space into fsaverage6 space. Correspondence is assessed as the Dice coefficient, where 0 is no spatial correspondence and 1 is total spatial correspondence. In order to assess statistical significance, a spin test^20,31,32 implemented in Python³³ is performed with 1000 permutations for each reference atlas network.

The NCT can be installed using the command pip install cbig_network_correspondence. Figure 6 illustrates the usage of the NCT to explore network correspondence between user-defined input data and a set of atlases. Specifically, users specify the pathway to the input data and provide a configuration file containing input data details such as the data name, data space, and data type. Additionally, users provide a list of atlases included in the NCT, available at https://github.com/rubykong/cbig_network_correspondence#atlases-we-included.

Fig. 6: Illustration of NCT usage to explore network correspondence between user-defined input data and a set of atlases.

In this example, we explore the overlap between a single-dimension input dataset and 4 atlases: the Yeo2011 17-network atlas (“Yeo2011-17”, the Gordon2017 17-network atlas (“Gordon2011-17”), the Glasser2016 360-ROI atlas with Ji2019 12 Cole-Anticevic networks (“Glasser2016-360 + Ji2019-12”), and the Shen2013 268-ROI atlas with 8 networks (“Shen2013-268-8”). A The user provides the input data together with a config file specifying the name, data space (e.g., fs_LR_32k, fsaverage6, FSLMNI2mm), and data type (“Metric” if the input data contains floating values; “Hard” if the input data contains binary values). In this example, the input data is in fs_LR_32k space and contains binary values. The data type is “Hard”. The user also provides an atlas list indicating which atlases to include. B The NCT reads the input data and projects the atlases in the atlas list to the input data space. C The NCT computes the Dice overlap between the input data and networks from the atlases in the list. The NCT also performs spin tests to test whether the input data and networks significantly overlap. D The NCT reports the network correspondence summary using a “Network Clock” plot, a “Network Radar” plot, and a summary table for single-dimensional data. The “Network Clock” provides a visual comparison of the Dice overlap across networks from different atlases. Different colors represent different atlases. The lollipop bars represent the Dice overlap coefficients. Networks significantly overlapping with the input data (p < 0.05) are indicated by the network names. A larger font size represents a larger Dice coefficient. The “Network Radar” shows the Dice overlap across networks within each atlas. Networks significantly overlapping with the input data (p < 0.05) are indicated by “*”. The NCT also provides a summary table showing the exact Dice coefficients and p-values across different atlases (see Supplementary Table 3). The NCT uses network names from the original paper for each atlas. Dors dorsal, attn attention, par parietal, ven ventral, sal salience, subcor subcortical, cing cingulo, operc opercular, vent ventral, multi multimodal, orbit orbital, front frontal.

The NCT proceeds by first reading the input data, projecting the atlases specified in the list to the input data space, computing the spatial similarity between the data and atlases, and conducting statistical analyses. The network correspondence results are then visually presented.

The input data can be either uni-dimensional or multidimensional. Examples of unidimensional data include a spatial map of a single ICA component, a task contrast map, or a single network from a hard parcellation. The network correspondence results of unidimensional data are illustrated as a “Network Clock” plot, a “Network Radar” plot, and a summary table (Fig. 6D). The “Network Clock” provides a visual comparison of the Dice overlap between the input data and networks from different atlases. Different colors represent different atlases. The lollipop bars represent the Dice overlap coefficients. Networks significantly overlapping with the input data (p < 0.05) are indicated by the network names. In this plot, networks that have larger Dice coefficients are shown by larger font sizes. The “Network Radar” shows the Dice overlap across networks within each atlas. Networks significantly overlapping with the input data (p < 0.05) are indicated by “*”. The NCT provides a “Summary Table” showing the exact Dice coefficients and p-values for different atlases.

In the case of multidimensionality, examples could include a set of ICA spatial maps, multiple task contrast maps, or any atlas included in the NCT. The NCT reports the network correspondence results of multidimensional data as “Overlap Heatmap” plots, where the rows correspond to the input data, and columns correspond to the networks of a comparison atlas. The k-th row and m-th column represents the Dice overlap coefficient between the k-th dimension of the input data and the network m from the atlas. In this plot, a brighter color indicates higher overlap, while a darker color indicates lower overlap. Networks significantly overlapping with the input data (p < 0.05) are indicated by “*”. The NCT also provides a summary table showing the exact Dice coefficients and p-values.

The NCT was built in Python and is available at the GitHub repository: https://github.com/rubykong/cbig_network_correspondence. The toolbox has been released to the Python Package Index (PyPI): https://pypi.org/project/cbig-network-correspondence. Detailed documentation and a tutorial are available at: https://rubykong.github.io/cbig_network_correspondence.

Illustrating the utility of the NCT with several examples

To demonstrate various applications of the NCT in both straightforward and challenging cases, we next show multiple network correspondence results. For the following examples, the image from an analysis (e.g., task activation map) was loaded, the space of the image set to a specific standard space (e.g., MNI152), and a threshold set for significant voxels. Dice coefficients were then determined for all reference atlases, and output as a Dice coefficient array. For each network within each atlas, we determined the Dice coefficient of network overlap and used spin tests to quantify levels of significance.

For the first example, we used the HCP working memory task fMRI contrast (2BK-0BK) which includes data from 997 HCP S1200 subjects³⁴. As illustrated in the “Network Clock” depicted in Fig. 7, this task fMRI contrast (top left) shows significant spatial overlap with several networks across multiple atlases (e.g., “FrontPar” network from the Glasser and Gordon atlases, “DorsAttn” network from the Laumann atlas). As shown in the “Network Radar” plot (top right), the NCT can be further used to quantify the overlap between user-defined maps and specific networks within an atlas. In the example highlighted in this figure, the input contrast shows significant overlap with the network labels “FrontPar” and “DorsAttn” in the Gordon atlas. This example illustrates a case where the task activation map corresponds well with networks specified in multiple atlases.

Fig. 7: Illustration of usage of the NCT to explore network correspondence between user-defined input data and multiple atlases: HCP Example 1.

In this example, we explore overlap between the HCP working memory task contrast (2BK vs. 0BK) and networks from 8 atlases. The user provides the input data together with a config file specifying the name, data space, data type, and the threshold which is used to threshold the input data. The user also provides an atlas list indicating which atlases to include. The NCT thresholds the data based on the config file and projects the atlases in the atlas list to the input data space. The NCT computes the Dice overlap between the thresholded data and all networks from atlases in the list. The NCT also performs spin tests to test whether the thresholded input data and networks significantly overlap. The NCT reports the network correspondence summary using a “Network Clock” plot (middle), a “Network Radar” plot (top right), and a summary table showing the exact Dice coefficients and p-values for all atlases (see Supplementary Table 3). The “Network Clock” provides a visual comparison of Dice overlap across networks from different atlases. Different colors represent different atlases. The lollipop bars represent the Dice overlap coefficients. Networks significantly overlapping with the thresholded input data (p < 0.05) are indicated by the network names. Larger font sizes represent larger Dice coefficients. The “Network Radar” shows the Dice overlap across networks within each atlas. Networks significantly overlapping with the thresholded input data (p < 0.05) are indicated by “*” in the “Network Radar”. WM working memory, BK back, dors dorsal, attn attention, par parietal, ant anterior, sal salience, LECN left executive control network, RECN right executive control network, DMN default mode network, ven ventral, subcor subcortical, mot motor, vis visual, cog cognition, occ occipital, front frontal, cing cingulo, operc opercular, med medial, lat lateral, ant anterior, MTL medial temporal lobe, SM somatomotor, post posterior.

For the second example, we used the HCP social task fMRI contrast (TOM-random) which includes data from 997 subjects. As illustrated in the “Network Clock” depicted in Fig. 8, this task fMRI contrast (top left) shows significant spatial overlap with several networks across multiple atlases (e.g., “VentAttn” network from the Laumann atlas, “Language” network from the Gordon atlas). As shown in the “Network Radar” plot (top right), the NCT further can be used to quantify the overlap between user-defined maps and specific networks within an atlas. In the example highlighted in this figure, the input contrast shows significant overlap with the network labels “Language” and “Context” in the Gordon atlas. This example illustrates a case where the task activation map does not correspond well with networks specified in multiple atlases. It thus signals that any network labels should be applied to this map with great caution.

Fig. 8: Illustration of NCT usage to explore network correspondence between user-defined input data and multiple atlases: HCP Example 2.

In this example, we explore the overlap between the HCP social task contrast (theory of mind vs. random) and 8 atlases. The user provides the input data together with a config file specifying the name, data space, data type, and the threshold which is used to threshold the input data. The user also provides an atlas list indicating which atlases to include. The NCT thresholds the data based on the config file and projects the atlases in the atlas list to the input data space. The NCT computes the Dice overlap between the thresholded data and all networks from atlases in the list. The NCT also performs spin tests to test whether the thresholded input data and networks significantly overlap. The NCT reports the network correspondence summary using a “Network Clock” plot (middle), a “Network Radar” plot (top right), and a summary table showing the exact Dice coefficients and p-values for all atlases (see Supplementary Table 4). The “Network Clock” provides a visual comparison of Dice overlap across networks from different atlases. Different colors represent different atlases. The lollipop bars represent the Dice overlap coefficients. Networks significantly overlapping with the thresholded input data (p < 0.05) are indicated by the network names. Larger font sizes represent larger Dice coefficients. The “Network Radar” shows the Dice overlap across networks within each atlas. Networks significantly overlapping with the thresholded input data (p < 0.05) are indicated by “*” in the “Network Radar”. Dors dorsal, attn attention, post posterior, multi multimodal, temp temporal, par parietal, med medial, front frontal, ven ventral occ occipital, lat lateral, vis visual, cing cingulo, operc opercular, ant anterior, MTL medial temporal lobe, SM somatomotor.

For the next example, we used the UK Biobank ICA component 5 to illustrate how the NCT can facilitate reporting of results from this type of analysis. As illustrated in the “Network Clock” depicted in Fig. 9, this ICA map (top left) shows significant spatial overlap with networks labeled “Visual” across multiple atlases. The final example (Fig. 10) uses UK Biobank ICA component 3 as the input map. Even with this complex input spanning multiple brain areas, the NCT can be used to quantify overlap with existing network labels to facilitate communication and comparison. Again, these two examples illustrate cases in which the input maps correspond well (Fig. 9) and do not correspond well (Fig. 10) with networks specified in multiple atlases.

Fig. 9: Illustration of NCT usage to explore network correspondence between user-defined input data and multiple atlases: UKB Example 1.

In this example, we explore overlap between UKB ICA component 5 and 8 representative atlases. The UKB ICA z-stat maps with 21 good components (https://www.fmrib.ox.ac.uk/ukbiobank/group_means/rfMRI_GoodComponents_d25_v1.txt) were thresholded by FSL melodic mixture-modeling threshold 0.6. The component 16 corresponds to the cerebellum and was further excluded, resulting in 20 thresholded UKB ICA maps. The 20 thresholded UKB ICA maps can be found in Supplementary Fig. 1. The NCT also provides a summary table showing the exact Dice coefficients and p-values for all atlases (Supplementary Table 5). Med medial, vis visual, lat lateral, prim,primary, cing cingulo, operc opercular, SM somatomotor, dors dorsal, attn attention, par parietal, front,frontal, post posterior, ant anterior, MTL medial temporal lobe.

Fig. 10: Illustration of NCT usage to explore network correspondence between user-defined input data and multiple atlases: UKB Example 2.

In this example, we explore overlap between UKB ICA component 3 and 8 representative atlases. The UKB ICA z-stat maps with 21 good components (https://www.fmrib.ox.ac.uk/ukbiobank/group_means/rfMRI_GoodComponents_d25_v1.txt) were thresholded by FSL melodic mixture-modeling threshold 0.6. The component 16 corresponds to the cerebellum and was further excluded, resulting in 20 thresholded UKB ICA maps. The 20 thresholded UKB ICA maps can be found in Supplementary Fig. 1. The NCT also provides a summary table showing the exact Dice coefficients and p-values for all atlases (Supplementary Table 6). Cing cingulo, operc opercular, dors dorsal, attn attention, mot motor, vis visual, assoc association, sal salience, ven ventral, post posterior, front frontal, par parietal; SM somatomotor, ant anterior, MTL medial temporal lobe, lat lateral, med medial.

Taken together, these examples illustrate how researchers can use the NCT to quantitatively and accurately present novel cognitive and network neuroscience results in reference to widely used atlases to enhance interpretability and comparison with existing findings in the literature.

Discussion

Despite decades of functional neuroimaging research resulting in repeated and robust observation of multiple large-scale functional brain networks, the nomenclature describing these networks in humans is not standardized⁵. Instead, there has been a proliferation of different population-average network atlases that rely on distinct nomenclatures derived by independent research groups using disparate approaches⁸. Indeed, we have seen that even networks with similar nomenclature (e.g., “salience”) across atlases may not share the same spatial topography.

Although atlases vary both in their number of networks and the methods used to identify networks, they all seek to categorize network affiliations for the cortical mantle and subcortex. The goal of each OHBM Best Practices committee is to formalize and endorse best practices across domains of human neuroimaging research to promote science literacy, transparency, and reproducibility. Building on the consensus conclusions from the OHBM Best Practices committee on network nomenclature (WHATNET), here we introduce the NCT.

In building the NCT we first investigated the spatial correspondence of discrete large-scale functional brain networks defined in sixteen widely used atlases from a variety of standardized stereotaxic spaces (volume images and surface maps), network definition methods, and datasets. Overall, a high degree of spatial correspondence across atlases was observed for many functional networks. However, areas of notable divergence were also observed, especially for networks spanning association cortices. We further quantitatively interrogated convergence directly across two widely used atlases (Yeo2011-17 and Gordon2017-17). These analyses revealed that a network presenting as unitary in one atlas was in some cases broken down into two networks in the other atlas. Network nomenclature across the two atlases was also inconsistent in many cases. Such discrepancies support the published 2023 recommendations of WHATNET advising that network-based interpretations of novel activation or functional connectivity findings use multiple reference atlases produced by independent research laboratories to corroborate network localization⁸. The NCT is designed to further promote and facilitate the adoption of this recommendation by providing researchers with a user-friendly tool to readily conduct and report on network correspondences, as demonstrated with multiple examples. We suggest that future studies reference multiple independent atlases to quantitatively evaluate network localization of activity or functional connectivity, using the NCT^13,35. This would support cross-study comparisons for replicability, aid in the interpretation of novel findings, and encourage greater dialogue across research groups and the field of network neuroscience more broadly.

Many novel discoveries in cognitive and network neuroscience are contextualized within the bounds of previously identified network topographies. Methods for assessing network overlap with novel functional connectivity or fMRI task activation maps vary from visual inspection, Euclidean distance between network regions’ center of mass and activation peaks, to direct quantification between an atlas and neuroimage. Variability in image formats between atlases and result images can be a significant obstacle to assessing correspondence directly and/or empirically. Many fMRI task studies are conducted in volume space, yet some atlases are only shared by the creators in surface space^24,36, arguing that volume rendering is less precise³⁷. In order to surmount this problem, all atlases included in the NCT were transformed from their original standard space to be interoperable and compatible with all other standard spaces, in the volume or surface. This approach was integral to assessing the correspondence between all atlases. Further, this approach now allows for a direct and empirical examination of novel neuroscience findings against multiple existing widely used atlases simultaneously.

Previous work has attempted to quantify spatial variability among major large-scale brain networks as defined in widely used functional atlases, also noting only modest similarity between atlases. Some have created atlas-based labeling tools³⁸, but none have provided a means for comparing novel results with multiple atlases simultaneously. One project created a Consensual Atlas of REsting-state Network based on Yeo, Gordon, and Doucet atlases³⁹. Initial efforts to standardize human brain parcellations have resulted in tools such as Neuroparc, which provides a means for comparison among widely used atlases⁴⁰. Others have compared anatomical, functional connectivity, and random parcellations and concluded that there is no optimal approach for choice of parcellation technique⁴¹. Recent parcellation evaluation studies have suggested that multi-modal parcellations combining functional and anatomical metrics may perform worse than those based only on functional data because functionally homogeneous brain regions can span major anatomical landmarks⁴². Critically, these studies have either simply evaluated the correspondence between different atlases or attempted to derive a single ‘consensus’ atlas. The NCT obviates the need to rely on any single atlas. Instead, it transparently addresses the inherent ambiguity in assigning network labels to empirical brain maps by objectively quantifying the spatial correspondence between a given empirical map and multiple existing, widely used atlas nomenclatures simultaneously.

It should be noted that all neuroimaging-derived spatial maps are sensitive to thresholding choices. The issue of appropriate thresholding of fMRI results is outside the scope of the current work, but several previous discussions of this topic are available, and guidelines regarding thresholding are put forth in early OHBM Best Practices reports⁶. We recommend unthresholded image deposition in an open access repository (e.g., Neurovault https://neurovault.org/) so that researchers can come to their own conclusions regarding overlap between novel results and existing parcellation schemes.

It is important to acknowledge that the NCT does not determine for the user what level of correspondence should be considered sufficient to warrant a particular network label for a given result. The goal of providing the NCT to the community is to assist in mapping novel results to network nomenclatures adopted by multiple widely-used atlases. We do not wish to be prescriptive regarding how many matches a user should pick for any given set of results, as that depends entirely on the research question under investigation. However, as the Dice coefficient is a measure of effect size, which is reported at the same time as the p-value, both are integral for interpretation. We agree that p-values alone are never sufficient to determine the interpretation of findings. For this reason, exemplar reporting standards in the revised submission provide measures of both effect size and significance.

The WHATNET now recommends that new scientific reports on patterns of task activation or functional connectivity, if purported to engage a specific large-scale functional brain network, be indexed to multiple independent atlases for verification. The NCT provides an accessible, easy-to-use tool to follow this recommendation in full. Critically, we see the NCT as an evolving resource, wherein the databases of atlases will be open to the research community. This will enable the NCT, as a tool to quantify network correspondences, to expand as novel parcellation schemes and atlases are introduced. Tools like the NCT will be especially critical for integrating findings across psychiatry and neurology, where the lack of guidance regarding atlas correspondence and network nomenclature can be an impediment to clinical progress⁴³.

Methods

Functional brain atlases

We investigated cortical network correspondences between user-defined input data and a set of functional network atlases. While there are 23 atlases available in the most recent release of the NCT (v 0.3.1), we considered 16 widely used functional network atlases that are publicly available in fsaverage6 surface space, fs_LR_32k surface space, and MNI space including the Laird2011, Yeo2011, Shirer2012, Smith2013, Shen2013, Laumann2015, Miller2016, Gordon2016, Gordon2017, Schaefer2018, and Ji2019^{14,15,16,17,18,19,20,21,22,23,24,25} for the analyses depicted in Figs. 2 and 3. The majority of these atlases are derived from resting state fMRI data, though a few are based on task fMRI. Some of these atlases have multiple resolutions, resulting in 16 different atlases (Supplementary Table 1). We focused on cortical networks in this study; the non-cortical regions were masked for all atlases. Networks only corresponding to subcortical regions and the cerebellum were also excluded from all parcellations examined. A full list of currently available atlases in the NCT can be found in https://pypi.org/project/cbig_network_correspondence.

Network similarity between atlases

To compute the network similarity between the input data and the atlases included in NCT, we treat the input data space as the reference data space and project the atlases to the reference atlas space (see section Projection between atlases in different spaces for projection details). In the case of input data being an atlas, the input data will be referred to as the “reference atlas”. Figure 1 shows an example where the Yeo 17-network atlas in fsaverage6 space (the center) served as the reference atlas. All other surrounding atlases in different spaces were projected to the fsaverage6 space. The network similarity is defined as the Dice coefficients between the input data and different atlases in the reference data space.

For each reference network, we computed the Dice coefficient between this reference network and each network from all other atlases in the reference atlas space. Assuming there are K networks in the reference atlas and M networks in another atlas, we generated a KxM network similarity matrix, where the k-th row and m-th column represents the Dice overlap coefficient between network k of the reference atlas and network m from the other atlas. A high Dice coefficient indicates high spatial overlap between the two networks. The network similarity matrix was ordered so that each reference network was paired up with the network from the other atlas with the best overlap (i.e., highest Dice coefficient) along the diagonal of the matrix. Sixteen functional atlases included in this study yields 240 different network similarity matrices.

Consistencies of network names across atlases

To explore the agreement and disagreement of network labels across sixteen widely used atlases, we clustered the networks with high spatial overlap together. A hierarchical clustering algorithm²⁶ was applied onto the network similarity matrix to group all networks into 15 clusters, the average number of networks across atlases. We reordered the network similarity matrix based on the hierarchical clustering results. The hierarchical clustering results were also visualized as a chord map. We then manually examined network names within each cluster to identify network spatial topography and their labels.

Spin tests

We use the spin test^20,31,32 implemented in Python³³ to assess if the reference input data significantly overlaps with networks from different atlases. Specifically, the reference input data is randomly rotated 1000 times. For each network, we compute the Dice coefficient between the rotated input data and this network. For a network highly overlapping with the input data, we expect the Dice coefficient between this network and the rotated input data to be smaller than the Dice coefficient between this network and the original input data. The p-value is defined as the number of rotated input data whose Dice coefficients are larger than the original input data divided by the total number of rotations. However, the spin test only works for data in surface space. For volumetric data, we project the data to fsaverage6 using an advanced nonlinear mapping approach⁴⁴ and perform the spin test in fsaverage6 surface space.

Projection between atlases in different spaces

For atlases included in NCT, we treated each atlas as the reference atlas and projected other atlases to the reference atlas space.

• 1. Volume to volume projection. If the reference atlas and the other atlas were in different volumetric spaces, we used antsRegistration to register the other atlas template to the reference atlas template.

• 2. Volume to surface projection. If the reference atlas was in fsaverage6 surface space and the other atlas was in volumetric space, we first projected the other atlas into MNI152 volumetric space and then used an advanced nonlinear mapping approach⁴⁴ to project the other atlas from MNI152 to fsaverage6 surface space. If the reference atlas was in fs_LR_32k surface space⁴⁵ and the other atlas was in volumetric space, we first projected the other atlas into fsaverage6 surface space as above and then projected it from fsaverage6 surface space to fs_LR_32k surface space using the fsaverage-to-fs_LR registration provided by HCP (https://wiki.humanconnectome.org/download/attachments/63078513/Resampling-FreeSurfer-HCP_5_8.pdf).

• 3. Surface to volume projection. If the reference atlas was in volumetric space and the other atlas was in fsaverage6 surface space, we first projected the other atlas to MNI152 space using an advanced nonlinear mapping approach⁴⁴ and then projected it from MNI152 to the reference volumetric template using antsRegistration. If the reference atlas was in volumetric space and the other atlas was in fs_LR_32k surface space, we first projected the other atlas to fsaverage6 surface space as above and then projected it from fsaverage6 to fs_LR_32k using the fsaverage-to-fs_LR registration provided by HCP.

• 4. Surface to surface projection. If the reference atlas was in fsaverage6 surface space and the other atlas was in fs_LR_32k surface space, we projected the other atlas to fsaverage6 surface space using the fs_LR-to-fsaverage registration provided by HCP. If the reference atlas was in fs_LR_32k surface space and the other atlas was in fsaverage6 surface space, we projected the other atlas to fs_LR_32k surface space using the fsaverage-to- fs_LR registration provided by HCP.

HCP and UKBiobank data examples

HCP and UKB group ICA spatial maps were thresholded by fitting a spatial mixture model to the histogram of intensity values in each independent component spatial map. Mixture modeling explicitly models the null (background noise) and signal (“activated” or “deactivated”) parts of the spatial map. A threshold value of 0.6 was set to give slightly more importance to identifying signal over noise. In practice, the resulting thresholded spatial maps are robust to small deviations (e.g., threshold value = 0.5, which places equal importance on labeling a voxel as signal vs noise, and higher result in spatial maps that are very similar).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.