Introduction

The ability to regulate emotional reactivity amidst the pursuit of task goals is a core pro-social function and is intimately tied to psychiatric well-being and illness1. As such, one of the most widely administered and studied tasks using functional magnetic resonance imaging (fMRI) is the emotional face-matching task (EFMT)2,3. Since its introduction in 2002, the EFMT has been used in over 250 fMRI studies to examine the neurobiological basis of emotion processing, as well as potential links between emotion-related brain activity and neuropsychiatric phenotypes4. Early studies using the EFMT primarily focused on group-level, task block-averaged brain activity2,3. These studies established robust maps of the brain regions whose activity, for instance, was greater during face-matching trials than shape-matching trials5, with the majority of these focused on amygdala activation and amygdala connectivity2,6. This task contrast method has proven powerful in identifying brain regions and networks whose activity significantly changes during emotional face processing relative to a face-stimulus-free baseline. However, it constrains the identification of potentially relevant brain activation patterns7 to those in the experimenters’ predefined model of how the measured fMRI signal is modulated by the task. Moreover, the reliance of this approach on averaged brain activity obscures fine-grained temporal dynamics that might occur within and between task blocks or conditions7.

More recent approaches to analyzing neural activity during the EFMT, including multivoxel pattern analysis8, meta-analytic clustering9, Bayes factor analysis10, and connectome-based predictive modeling11, have expanded the understanding of emotion processing beyond classically implicated structures like the amygdala to distributed large-scale networks12,13,14. These include networks that sense and integrate emotionally salient environmental signals, such as sensory networks, the ventromedial components of the default mode network (DMN)11, and the salience network15, as well as networks implicated in the top-down regulation of attention, cognition and behavior, such as left and right frontoparietal networks (FPN)16 and dorsal attention network (DAN)17. However, even those more sophisticated methods fundamentally rely on task contrast data as input. As such, the fine-grained temporal dynamics of these networks that may occur within and between blocks remains largely unexplored. Moreover, the top-down, model-constrained nature of task contrast data may obscure task-relevant activity with nuanced dynamics.

Here, as an alternative approach, we apply tensor independent component analysis (tICA18) to the EFMT to generate a bottom-up, data-driven search for brain activity whose time course aligns with one or more aspects of the task stimulation time course, and which thereby retains fine-grained temporal information. tICA is a well-validated18 unsupervised learning technique that can detect and trace the moment-by-moment activity of large-scale, distributed brain networks during task performance. Compared to other methods for examining brain temporal dynamics, tICA allows for separable tracking of concurrent brain processes (whereas methods like coactivation pattern analysis19 assume a single aggregate process at each time point) at frame-wise resolution (methods like sliding window correlation20 average activity over multiple frames). However, previous applications of tICA to task fMRI have focused primarily on assessing subject differences in ref. 21 or methodological optimizations yielded by ref. 22 tICA-identified networks, rather than on deeply characterizing the neurobiological features and functions of brain networks during task performance. Given that the EFMT has been included in multiple large-scale neuroimaging datasets, including the Human Connectome Project (HCP), UK Biobank, and in several of the Connectomes Related to Human Disease specifically as a probe of the NIMH Research Domain Criteria (RDoC) based Negative Valence System (NVS)23, and is used frequently to assess individual differences in negative affect, applying tICA to the EFMT stands to appreciably deepen the understanding of brain network dynamics and their relation to cognitive and psychological factors related to performance of this important task.

In the current study, we applied tICA to EFMT fMRI data from the HCP Young Adult sample to detect and trace the activity of large-scale brain activation patterns during different temporal sub-domains of the task (e.g., during the onset of emotional face blocks, throughout emotional face blocks, during the offset of blocks, in between emotional face blocks and neutral shape blocks, etc.). Thus, we used tICA to identify—and characterize at high spatiotemporal resolution—EFMT-recruited neuronal processes that may be missed or obscured by the constraints of contrast-focused analyses5. We also used tICA to generate individual subject loadings for each identified brain network, to assess how individual differences in network activity relate to individual differences in task performance, as well as cognitive/behavioral and mental health factors.

Results

The EFMT reproducibly recruits 10 large-scale networks

The HCP sample was split into two groups, Group 1 (n = 413) for network identification and a replication sample (Group 2; n = 416), with all members of a family included in the same group to prevent family structure-related data leakage across the two groups. Two runs from each participant (with right-left and left-right phase encoding, hereafter referred to as RL/LR) were analyzed separately. Run two (i.e., LR) displayed greater reproducibility in tICA-identified networks across Groups 1 and 2 than run one (i.e., RL), as indexed by the strength of both spatial map correlations and temporal course correlations. Specifically, nine of 10 spatiotemporally distinct large-scale brain networks recruited by the EFMT in Group 1’s second run (i.e., LR run) (Fig. 1) were robustly replicated in Group 2’s second EFMT run, whereas two of the 10 networks did not replicate across the first EFMT runs for each group (i.e., the RL runs) (Table 1). This suggests that network engagement may be sensitive to task practice, habituation, or fatigue effects within the scanning session. Given these first run versus second run differences, we focused subsequent analyses on the more robustly replicable run two (LR) data.

Fig. 1: Ten large-scale brain networks engaged during the EFMT.
figure 1

The spatial maps and temporal dynamics of EFMT-recruited networks identified by tICA. Hierarchical clustering based on the network time courses distinguished networks preferentially activated for faces > shapes and for shapes > faces. Colored lines show the temporal courses for networks identified in Group 1 LR, and dotted lines show the temporal courses of the corresponding networks identified in Group 2 LR. Each color denotes a different network. Underlying data are presented in Supplementary Data 1.

Full size image
Table 1 Reproducibility of EFMT-recruited networks across groups and runs
Full size table

The EFMT recruits most of the cortex and circumscribed non-cortical areas

Next, we examined the brain-wide spatial coverage and overlap of the EFMT-recruited networks. To do so, we first thresholded each network map using MELODIC’s Gaussian-Gamma mixture modeling alternative hypothesis testing approach with p = 0.524, which places equal weight on identifying signals vs noise. Voxels with a probability of being in the active class (modeled by the Gamma densities), or signal, that exceeds the probability of being in the noise class (Gaussian density) survive the threshold. We then binarized each voxel as being either suprathreshold (i.e., 1) or subthreshold (i.e., 0), and summed these binarized maps across the 10 networks. This analysis revealed that all 10 EFMT-recruited networks included the fusiform gyrus and higher-level visual cortex as nodes (Fig. 2). Nine of the 10 networks included the right pars triangularis of the inferior frontal gyrus (BA45), and seven of the 10 included primary somatosensory cortex (BA1). Collectively, 74% of the cortex was recruited as a node in one or more of the large-scale networks during the EFMT. Notably absent as a node in any of the 10 networks was the subgenual cingulate cortex as defined by Mayberg et al.25, a brain region strongly implicated in negative affect regulation and depression26,27. Outside of the cortex, seven of the 10 networks included the posterior cerebellum as a node, and six of the 10 included the amygdala. Other than the amygdala, subcortical recruitment was sparse, with parts of the striatum, thalamus, and hippocampus involved in only one network each.

Fig. 2: Spatial overlap of brain networks recruited during EFMT.
figure 2

A global view of brain involvement in the EFMT. The positive and negative components of each network from Group 1 LR were thresholded at p = 0.5 (mixture modeling), binarized, combined, and summed. Unshaded (i.e., greyscale) regions did not appear as nodes in any of the EFMT-recruited networks. This network overlap map strongly replicated in Group 2 LR (r = 0.91, p < 0.001).

Full size image

To inform the potential role of each task network in task performance, spatial overlap between the EFMT networks and canonical brain networks that have been previously reported was also assessed. Overlap between each network and each of 17 canonical large-scale networks defined in the Gordon et al. parcellation28 was computed using the network correspondence toolbox (NCT)29 (Fig. 3).

Fig. 3: Overlap of EFMT networks with canonical networks.
figure 3

Heatmap displays Dice coefficients generated by the NCT29 quantifying the extent of spatial overlap between each EFMT-recruited network (Group 1 LR) and each of the 17 canonical large-scale networks defined in the Gordon et al. parcellation28. Dice coefficients were thresholded at p < 0.05 using a spin test29. Findings were consistent in Group 2 LR. Networks include: default mode (Default); medial and lateral visual (MedVis, LatVis); cingulo-opercular (CingOperc); salience; frontoparietal (FrontPar); dorsal attention (DorsAttn); hand, face, and leg somatomotor (HandSM, FaceSM, FootSM); auditory; premotor; parietal memory (ParMemory); contextual association (Context); and anterior and posterior medial temporal networks (AntMTL, PostMTL).

Full size image

EFMT-recruited networks have diverse temporal dynamics

Hierarchical clustering based on the network time course correlation matrix (Supplementary Fig. 1) revealed two broad types of networks: those preferentially activated during emotional faces trial blocks (Fig. 1, left panel) and those preferentially activated during neutral shapes trial blocks (Fig. 1, right panel). This clustering is further reflected in the correspondence between the network time courses and EFMT stimulation time courses, identified by the post hoc fitting of network time series data with a standard block design general linear model (GLM; Fig. 4). At deeper levels of the clustering hierarchy, fine-grained temporal dynamics revealed several further distinctions. For instance, while the engagement of networks 1, 2, and 4 peaked in the middle of emotional face blocks and began tapering off at the end of these blocks, the engagement of networks 6 and 10 ramped up across emotional faces trials and peaked at the end. Meanwhile, network 7 engagement peaked at the onset and offset of emotional faces trials. The time course of nine of the 10 networks replicated strongly across groups (Fig. 1).

Fig. 4: Correspondence of brain network time courses with EFMT stimulation time courses.
figure 4

Network time courses identified by tICA were fit post hoc with a standard block design general linear model, showing that our data-driven approach identifies networks modulated by the task. Networks shaded red predominantly activated during face trials, while those shaded blue predominantly activated during shapes trials. Different hues denote different subgroups identified in the hierarchical clustering tree.

Full size image

Individual differences in EFMT-recruited networks and task performance

All individual differences analyses controlled for age, sex, and education and corrected for covariance in family structure (i.e., presence of siblings and twins in the study) using permutation analysis of linear models (PALM)30 with multi-level exchangeability blocks31. Three participants (two from Group 1 and one from Group 2) were removed from analyses examining brain-behavior associations due to low overall EFMT accuracy (<70%).

Given very high levels of accuracy on both faces and shapes conditions (Supplementary Fig. 2), leading to potential ceiling effects for these metrics, we operationalized EFMT performance by reaction time slowing on emotional faces trials relative to shapes trials, i.e., Rtfaces – Rtshapes, indexing emotion interference (Supplementary Fig. 3). The emotional face condition elicits an emotional reaction from participants that requires early attentional screening and downregulation of salient yet extraneous emotional information, to focus on the task-relevant aspects of the stimulus (i.e., matching the face identity). The difference in reaction time on emotional face trials versus neutral shapes trials thus gauges the extent of this emotion interference process. Individual differences in this measure of emotion interference were significantly associated with individual differences in the engagement of four networks at p < 0.005 (Bonferroni-corrected for multiple comparisons), and of seven networks at the uncorrected level (p < 0.05) in Group 1. These significant associations were all in the positive direction, with the exception of Network 7. Relationships for Networks 2 and 10 were the most robust, replicating at the Bonferroni-corrected level in Group 2 (Fig. 5). The negative association observed for Network 7 did not replicate in Group 2.

Fig. 5: Individual differences in brain network engagement and EFMT task performance.
figure 5

For Group 1 (n = 411 subjects) and for the Group 2 replication sample (n = 415 subjects), the relationship between subject loadings on each large-scale network and reaction time slowing on emotional face trials (i.e., emotion interference), controlling for age, sex and education, and correcting for covariance in family structure using permutation analysis of linear models (PALM). Networks with significant relationships in both groups that are below the Bonferroni threshold for multiple comparisons correction (p < 0.005) are bolded and italicized; networks with significant relationships in both groups below at least p < 0.05 are bolded. Underlying data are presented in Supplementary Data 2.

Full size image

To further test the robustness of the relationship between EFMT-recruited brain networks and emotion interference, we examined the ability of a machine learning regression model trained on Group 1 to make individualized predictions in Group 2 of emotion interference based on brain network subject loadings. We used a random forest regression model for its ability to handle complex datasets with diverse feature types32. Using only subject loadings on the 10 EFMT-recruited brain networks, the regression model generated subject-specific predictions of emotion interference that significantly correlated with the actual values, r = 0.15, p = 0.004; MSE = 0.426. We subsequently applied SHAP (SHapely Additive exPlanations)33 to the predictions made by the random forest model trained on Group 1 and applied to Group 2 to quantify the feature importance of each EFMT-recruited brain network for predicting emotion interference. In line with the correlation results, Network 2 had the highest feature importance for this analysis; this was followed by Network 7 (Fig. 6). High subject loadings on Network 2 and low subject loadings on Network 7 predicted high levels of emotion interference.

Fig. 6: Feature importance of EFMT-recruited brain networks for predicting emotion interference.
figure 6

Features are ordered from top-to-bottom on average importance. Each dot represents a subject’s SHAP value for a given network. Dots colored red denote data points that increase the prediction for a given network, while dots colored blue denote the data points that decrease the prediction for that network. Underlying data are presented in Supplementary Data 3.

Full size image

Individual differences in EFMT-recruited networks and cognitive/mental health factors

To then examine relationships between EFMT-recruited brain networks and latent psychobehavioral phenotypes, we first applied the five-factor solution characterized by Schottner et al.34 derived from hierarchical factor analysis of 87 variables from across the HCP’s assessments of alertness, cognition, emotion, motor function, personality, sensory function, psychiatric and life function, substance use and in-scanner tasks, to compute subject scores on the five factors: well-being/positive affect, internalizing/negative affect, processing speed, cognition, and substance use34.

Individual differences in the engagement of four different networks (1, 2, 6 and 10) were reproducibly associated with individual differences in cognition scores across both groups (Fig. 7). However, the random forest machine learning regression model trained on Group 1 was not able to generate accurate subject-specific predictions of cognition based on network subject loadings in Group 2 (r = 0.037, p = 0.462, MSE = 1.12). This may be partly due to the influence of unaccounted variance in educational attainment, which is strongly associated with cognition. Indeed, a second model in which education was residualized out of the network loadings and cognition scores showed stronger prediction accuracy (r = 0.086, p = 0.090, MSE = 0.70).

Fig. 7: Individual differences in brain network engagement and cognitive/mental health factors.
figure 7

Group 1 (n = 411 subjects) relationships between subject loadings on each large-scale network and five facets of behavior derived by factor analysis from 87 HCP-YA behavioral and neuropsychiatric assessments: positive affect/well-being, cognition, negative affect/internalizing, processing speed, and substance use. Analyses control for age, sex, and education, and correct for covariance in family structure using permutation analysis of linear models (PALM). Significant relationships below the Bonferroni threshold for multiple comparisons correction (p < 0.001) are denoted. Underlying data are presented in Supplementary Data 4.

Full size image

Moreover, none of the networks were correlated with or predictive of individual differences in non-cognitive facets (i.e., positive affect/well-being, negative affect/internalizing, processing speed, or substance use). Notably, networks 4 and 9 were not associated with individual differences in cognition or emotion interference in either group, despite these networks’ strong temporal alignment with multiple sub-domains of the task (Figs. 1 and 4).

Given the surprising lack of association between EFMT-recruited brain networks and negative affect/internalizing or positive affect/well-given, we conducted additional analyses to investigate whether more specific subcomponents of these domains might relate to the EFMT brain networks. To do so, we used normalized scores on the 87 neurocognitive and neuropsychiatric HCP-YA assessments, scores on the 5 latent mental health domains, and scores of EFMT emotion interference, as inputs to the random forest machine learning model (trained on Group 1, tested on Group 2) to predict subject loadings on each of the 10 EFMT-recruited networks, and then applied SHAP to quantify the feature importance of all 93 assessments for each network (Fig. 8). While the top important features for predicting loadings on each network were dominated by subcomponents of cognition, several subcomponents of positive and negative affect were also linked to some networks. For instance, the top 3 most important features for predicting loadings on Network 3 were self-efficacy (positive affect), sadness (negative affect), and anger (negative affect). Anger was also in the top three of most important features for predicting loadings on Networks 7 and 9.

Fig. 8: Feature importance of neurocognitive and neuropsychiatric variables for predicting EFMT-recruited brain network loadings.
figure 8

Mean, absolute SHAP value of the top 9 most important features for predicting subject loadings on each of the 10 EFMT-recruited brain networks. Underlying data are presented in Supplementary Data 5. Full variable names and descriptions for each measure are listed in Supplementary Data 6.

Full size image

Comparisons to amygdala connectivity and activation during the EFMT

A central focus of prior research on the EFMT’s clinical utility has been on amygdala activation and connectivity13. To compare the task performance and psychobehavioral information reflected in amygdala connectivity and activation to that in the presently identified large-scale networks, we used dual regression35 of the full set of group tensor ICA spatial maps against each participants task fMRI data (for each run separately) to compute functional connectivity between the left and right centromedial amygdala, basolateral amygdala, and superficial amygdala and each of the 10 large-scale networks. We also extracted amygdala activation from the task activation spatial maps5 from the faces > shapes contrast computed using a standard voxel-wise general linear model that was released by the HCP. From these maps, we calculated the average regression weight in each of the six amygdala regions of interest for each participant. No relationships between amygdala connectivity or amygdala task activation and emotion interference or cognitive/mental health domains survived Bonferroni correction for multiple comparisons. Effect sizes for these relationships were also considerably smaller compared to the brain-behavior findings using the large-scale networks. As such, tICA-derived networks appear to be a more robust neural correlate of EFMT task performance and general cognition than amygdala connectivity or activation in relatively healthy adults.

Discussion

The present study reveals the utility of going beyond model-driven (i.e., GLM or multiple regression) contrast-based analyses of task fMRI data. Voxel-wise GLM analysis applies the same a priori specified model of the brain’s temporal response at every voxel, with resulting contrast maps (e.g., faces > shapes) reflecting an aggregate map comprised of any voxel significantly activated by the task, according to the a priori model. As shown in previous work7, this map vastly underestimates the extent of task-relevant neuronal activity, in part because brain regions and networks with temporal responses to the task that differ from the a priori model time courses will not be identified, and in part because the common subtraction paradigm to generate a contrast map removes brain activation common to both conditions. This neglects the dynamic relationships between brain areas and networks. Further, even if a subtraction map reflects a more spatially distributed pattern of brain activation, a GLM-based contrast map is an aggregate brain activation map that can reflect activity of an undifferentiated mix of temporally distinct networks. Here, by using tICA to resolve dynamic and temporally distinct task-related activity and networks, we uncover an appreciably larger-than-previously-considered engagement of the brain by the EFMT: 74% of cortex and circumscribed subcortical regions, including the amygdala and cerebellum.

We found that EFMT-recruited brain networks with greater activity during face matching relative to shapes (i.e., Networks 1, 2, 3, 4, 6, 7, and 10) have diverse temporal activation patterns, likely reflecting distinct aspects of task performance. For instance, Network 10 activity ramped up during face blocks, while Network 2 activity remained more constant during face blocks. Several networks also displayed changes in their activation patterns across the course of the task run. For instance, Network 3 displayed strong activation during the first shape block, but little to no activation during subsequent shape blocks. Such differences between early and later blocks were observed for multiple networks, despite our focus on the second task run, suggesting ongoing impacts of learning and novelty on brain network activation within runs, even after a full run of practice. These within-block and within-run network activation changes (Fig. 1) identified using the tICA data-driven approach are not captured by standard voxel-wise GLM analyses, which model each task condition as a block of constant activity convolved with a hemodynamic response function and may reflect overlooked (or difficult to model) effects of practice, habituation, or fatigue. Overall, tICA’s use of fine-grained temporal information identified a richer-than-expected tapestry of concurrent neurobiological processes recruited by the EFMT, distinguishing brain networks with diverse activation patterns and with distinct relations to task performance and general cognition.

The 10 EFMT-recruited networks identified by tICA involve the interaction of visual association cortex with different sets of non-visual brain regions in distinct temporal fashions. These diverse interactions are supported by the multifaceted anatomical connections between the visual cortex and the rest of the brain. Kravitz et al.36 detail the ventral visual network’s projections to at least six distinct areas that support different cognitive functions, including an occipitotemporal-amygdala pathway involved in detecting and processing emotionally salient stimuli and an occipitotemporal-VLPFC pathway supporting object working memory. Networks 2 and 6 show extensive recruitment of both the occipitotemporal-VLPFC pathway and the occipitotemporal-amygdala pathway. However, their time courses suggest distinct roles in task performance. Network 2 remains active throughout face blocks, peaking mid-block, suggesting its engagement of working memory processes that support maintenance of the task rule set despite the presence of salient emotional stimuli, supported by DAN-mediated spatial executive attention. On the other hand, Network 6 activity increases over the course of each face block and is distinguished from Network 2 by the suppression of medial visual networks engaged during low-level processing of visual stimuli; as such, Network 6 may support the focus of visual attention as the interference load created by the emotional faces accrues across the block.

Notably, Network 2 also had the highest feature importance for predicting emotion interference, showing high subject loadings on Network 2 predicted high levels of emotion interference across samples. Network 10 was also significantly and reproducibly associated with individual differences in emotion interference during the EFMT as well as with general cognition across both groups. Both networks are preferentially activated during face blocks, but with distinct time courses (Network 2 activity peaking in the middle of face blocks and Network 10 activity peaking at the end of blocks), suggesting distinct roles of each network in the emotion regulation process. Unlike Network 2, Network 10 showed increased activation over the course of the faces blocks, activating most strongly within the second half of each of these blocks. While also containing lateral visual areas, Network 10 primarily resembled a right-lateralized FPN component, suggesting a role for this network in top-down executive control, which may become increasingly recruited over the course of the more challenging task blocks. This interpretation is bolstered by the observation that working memory task performance was a top predictive feature of Network 10 loadings. Network 5 (DMN + contextual association network) was also significantly associated with emotion interference, but in contrast to Networks 2 and 10, it was preferentially activated during inter-block intervals and suppressed during face blocks, which aligns with the DMN’s role as a task-negative network. This suggests that the capacity to suppress internally-focused thoughts during active periods of the task, and/or the ability to prepare for upcoming blocks during rest periods between task conditions, can also impact key aspects of task performance.

Even though individual differences in the recruitment of the remaining seven networks were not robustly associated with variability in emotion interference, they almost certainly play roles in supporting other cognitive aspects of task engagement and completion, as supported by the significant relations between several of these networks and general cognition (Fig. 7). The emotional face-matching condition and shape-matching (control) condition differ in several aspects besides emotion content that require engagement of multiple cognitive processes. First, the face stimuli are considerably more complex than the ovals used in the shape-matching condition, requiring modulation of focus and cognitive effort. Further, visual processing of human faces by other humans employs specialized brain mechanisms that enable rapid holistic assessment of facial identity and emotion37, requiring recruitment of distinct visual processing streams by the two conditions. Finally, the emotional face condition requires early attentional screening and later downregulation of salient yet extraneous emotional information to focus on the task-relevant aspects of the stimulus (i.e., matching the face identity). Each of these cognitive aspects of the task likely follows distinct time courses and is differentially impacted by practice, fatigue, repetition priming, and habituation.

For example, of these networks, 1 and 4 show increased engagement of the ventral visual stream, amygdala, and VLPFC in conjunction with deactivation of DAN and the premotor network. The negative spatial component of Network 4 also resembled the action mode network38, posited to activate during goal-directed and externally focused tasks. As such, each network is situated to process emotionally salient stimuli and modulate attention. These networks may, in part, act as complements to Network 2 to support spatially-focused attention within the matching task, providing concurrent suppression of attention to non-relevant areas of the visual field.

The positive component of Network 9 showed significant overlap with both DMN and FPN, although in contrast to Network 10 (which was associated with emotion interference and cognition), the FPN regions of Network 9 were left-lateralized. Network 9 also dramatically differed from Network 10 in its time course, showing suppression early in faces blocks as well as early in the first shapes block, which decreased over the course of these blocks. Activation of Network 9 occurred primarily within the inter-block interval, suggesting this executive network might play a role in task switching. The time course of Network 5 closely resembled that of Network 9 despite limited spatial overlap. Network 5 primarily overlapped with the DMN, which is activated during unconstrained resting state periods in the absence of goal-directed activity. As such, suppression of Network 5 during the first shapes task block and the faces blocks likely reflects the need for increased task focus during these periods and a consequent reduction in internally-focused or self-reflective cognitive processes. The increased activation of this network during the inter-block intervals may likewise result from a temporary increase in such internal reflections.

For Networks 3 and 7, which were rapidly activated or suppressed during the between-block transitions, activation may reflect the brief rest period, the presentation of the written instruction card, and preparation for a change in task requirements. The overlap with the language network in both cases suggests that the written instructions do play a role in the recruitment of these networks. The recruitment of DMN nodes and suppression of premotor and somatomotor cortex within Network 7, along with deactivation of visual networks and DAN during the intertrial interval in both networks, may reflect the temporary reduction in task demands. The recruitment of the parietal memory network during these intervals in Network 3 may be due to the need to recall general task instructions. In addition to the intertrial period, these networks were also differentially engaged during the faces and shapes conditions, though the engagement of Network 3 during the first shape block resembled that observed during subsequent faces blocks, and the suppression of Network 7 during shapes blocks decreased over the course of the run. These patterns converge to suggest roles for these two networks in the general recruitment and redistribution of attentional resources during the task.

Perhaps most surprisingly, despite the EFMT’s clear recruitment of multiple brain networks and regions implicated in emotion processing and internal affective state (e.g., amygdala), none of the EFMT networks reflected individual differences in internalizing/negative affect or well-being/positive affect. This coincided with the absence of the subgenual cingulate, a brain region centrally linked to negative affect and depression26,27, as a node in any of the EFMT-recruited networks. This was particularly striking given the involvement of nearly 75% of the cortex in one or more of these networks. One possible reason for the absence of subgenual cingulate in these networks is the known fMRI signal dropout in this brain region due to its proximity to tissue-air boundaries39. However, other brain regions known to experience similar signal dropout were detected here (e.g., fusiform gyrus)39. Moreover, other studies of related in-scanner tasks that engage emotion interference have also failed to reliably detect subgenual cingulate activity40,41. Together with convergent findings from the systematic review of the EFMT literature13 and other recent studies42,43, our findings suggest that while the EFMT may engage emotion-related processes, it may not be salient, challenging, or evocative enough to be a truly incisive tool for assessing individual differences in the NVS constructs. Notably, although several large-scale neuroimaging studies include this task as an RDoC NVS probe, the task is actually not listed in the RDoC matrix as a standard probe for the NVS.

There are some limitations to the present study. First, while tICA is a purely data-driven technique, its use involves some subjective decision points as described in the Methods, including the choice of model order and the classification of noise and signal components. Second, while the tICA identified 10 task-related networks and their temporal profiles, the task performance data that are available are somewhat limited for interpreting the task components that may be engaging each network. Probing the roles of each of these networks by linking to more detailed performance measures will be an exciting new direction for research. A second limitation of our study is that the participants in the HCP study were generally healthy young adults, with no or only sub-clinical levels of anxiety, depression, and drug use. This may limit our ability to link brain networks to individual variability in negative affect. However, our findings are consistent with the review by Savage et al.13, which also found no consistent relationship between brain activation in primarily emotion-processing brain areas during EFMT and mental health disorder diagnoses, including major and bipolar depression, anxiety-related disorders, and obsessive compulsive disorder, among others. But given the wealth of insights into brain dynamics from tensor ICA, exploration of this task in clinical populations using the tensor ICA approach may reveal relationships between brain network dynamics and clinical measures.

In summary, we used tICA to identify and characterize spatiotemporal features of brain network processes recruited by the EFMT, revealing a rich dynamic landscape of brain activity that has not been observed using conventional contrast-based analyses of this important task. Our findings call into question the EFMT’s suitability to evoke brain activity that distinguishes individual differences in NVS function in healthy young adults, although more work is needed to determine whether brain dynamics during EFMT is of utility in clinical populations.

Methods

Subjects

Data used here included fMRI data from the Human Connectome Project Young Adult (HCP-YA) 1200 release. Participants provided informed consent to take part under the approval of the Washington University in St. Louis institutional review board. All ethical regulations relevant to human research participants were followed. A detailed description of the recruitment process for the HCP-YA is provided by others44,45. Briefly, individuals were healthy young adults, and exclusion criteria included a history of major psychiatric disorder, neurological disorder, or medical disorder known to influence brain function. For the present study, we additionally excluded subjects with HCP-YA quality assessment flags, subjects with excessive head motion (i.e., 1 or more frames with frame-wise displacement > 2 mm), and subjects without complete EFMT fMRI data. For reproducibility assessment, we randomly separated eligible subjects into two groups with non-overlapping family structure (Group 1: n = 413; Group 2: n = 416). Given the oversampling of siblings and twins in the HCP-YA cohort, we consolidated subjects from the same family within the same group to prevent family structure-related data leakage across the groups. While we did not rebalance the family-separated groups by other demographic variables in order to maintain clean splits to prevent data leakage (Table 2), we confirmed that there were no significant (p < 0.05) interactions between gender and network loadings in relation to task performance or mental health.

Table 2 Participant demographic data
Full size table

fMRI acquisition

HCP neuroimaging data were acquired with a standard 32-channel head coil on a Siemens 3T Skyra modified to achieve a maximum gradient strength of 100 mT/m45. T1-weighted high-resolution structural images, acquired using a 3D MPRAGE sequence with 0.7 mm isotropic resolution (FOV = 224 mm, matrix = 320, 256 sagittal slices, TR = 2400 ms, TE = 2.14 ms, TI = 1000 ms, FA = 8°), were analyzed using the HCP minimal pre-processing pipelines to register functional MRI data to a standard brain space. Blood oxygenation level dependent fMRI data were acquired using a gradient-echo EPI sequence with the following parameters: TR  =  720 ms, TE  =  33.1 ms, flip angle = 52°, FOV  =  280 × 180 mm, Matrix = 140 × 90, Echo spacing = 0.58 ms, BW  =  2290 Hz/Px, 72 slices, 2.0 mm isotropic voxels, with a multiband acceleration factor of 8. HCP neuroimaging data were acquired in two sessions, each occurring on a different day. Two, 2 min 16 s runs of the EFMT were acquired sequentially during session two, first with right-left (RL) phase encoding and then with left-right (LR) phase encoding.

fMRI pre-processing

The HCP fMRI minimal pre-processing pipeline is described in detail in Glasser et al.46. Briefly, minimal pre-processing of fMRI data corrects for spatial distortions, realigns volumes to compensate for subject motion, registers the fMRI data to the structural image, reduces the bias field, normalizes the 4D image to a global mean, and masks the data with a final FreeSurfer-generated brain mask46. We additionally applied 4 mm spatial smoothing, 150 s temporal smoothing, removal of non-steady-state frames (first 15 frames), and ICA-FIX automated denoising method24, which denoises residual subject motion, physiological motions, and artifacts, to the HCP minimally preprocessed EFMT fMRI data.

Emotional face-matching task (EFMT)

The EFMT, as implemented for HCP-YA, was adapted from Hariri et al.2. Subjects were presented with alternating blocks of trials that asked them to decide either (1) which of two faces presented on the bottom of a screen matched a face at the top of the screen, or (2) which of two shapes presented at the bottom of a screen matched the shape at the top of the screen (shapes were ovals, participants matched orientations). The faces all have either angry or fearful expressions. Each of the two EFMT runs included 3 face blocks and 3 shape blocks. Each block included 6 trials of the same stimulus type (i.e., faces or shapes), wherein each trial consisted of a 2 s stimulus presentation followed by a 1 s intertrial interval (ITI). Each block was also preceded by a 3 s task cue (face or shape), making each block 21 s in total.

Given a strong ceiling effect in matching accuracy on this task, we prioritized reaction times in assessing task performance. More specifically, to gauge individual differences in emotion interference (i.e., the extent to which processing emotional stimuli slows subjects’ ability to complete the task-matching goal), we targeted the difference in reaction time between face-matching trials and shape-matching trials as our primary task performance metric.

fMRI analysis

We applied tICA using MELODIC18,47 in FSL (version 6.0) to four separate runs of FIX-denoised EFMT fMRI data: Group 1 RL, Group 1 LR, Group 2 RL, and Group 2 LR. This identified task-relevant brain networks, including their voxel-wise spatial maps and their frame-wise time courses, and the individual subject loadings on each network, for each run in both groups. For each run, we ran tICA at varying model orders between d = 5 and d = 25 to inspect the output for the optimal model order for analysis. Because the goal was to investigate large-scale canonical brain networks, the model order of the group ICA needed to be relatively low to result in a coarse spatial scale parcellation of the brain into networks. Model orders that are too high produce finer spatial scale parcellations that reflect excessive segregation of known large-scale networks into individual or bilateral brain areas rather than networks. At the same time, model orders that are too low fail to fully disaggregate known canonical brain networks. Therefore, we tested a range of model orders from 5 to 25 to identify which decomposition parcellated the brain into canonical network patterns while effectively unmixing any noise/artifacts into separate components. For each model order, we examined the time course, power spectrum, and spatial map of each component to identify those most likely to reflect task-related brain activity (e.g., components with spatial patterns resembling large-scale brain networks and with temporal courses reflecting one or more aspects of the task time course). Ultimately, we identified d = 20 to be the optimal model order in our data that best achieved the balance between disaggregating large-scale canonical networks and not producing excessive segregation; this model order is also in line with previous reports examining the correspondence between task activation networks and canonical resting state brain networks48,49. Among the 20 components, those identified as noise (e.g., physiological, motion, scanner artifacts, drift) via visual examination of their time course, power spectrum, and spatial distribution were removed from analysis, in line with best practices for identification of ICA noise components50. The remaining brain network components were then quantitatively assessed for both spatial and temporal reproducibility across the RL and LR acquisitions and across Group 1 and Group 2. Ultimately, 10 of the 20 components were identified as brain networks that replicated across acquisitions and/or groups (Table 1). These 10 components were the focus of subsequent analysis. We applied hierarchical clustering to the correlation matrix generated as the average of the Fisher-z transformed correlations between each pair of the 10 identified EFMT-network components to distinguish groups of networks with overlapping and distinct time courses.

To examine correspondence between network time courses identified by tICA and the task stimulation time course (illustrated in Supplementary Fig. 4), the 10 network time courses were fit post hoc with a standard block design GLM with four regressors: face blocks and shapes blocks—each convolved with a gamma hemodynamic response function—and the temporal derivatives of each. A Bonferroni-corrected threshold of p < 0.0005 was applied to account for 10 networks and 10 contrasts of interest.

To further characterize the spatial maps of the 10 task networks, we used the NCT51 to compute overlap (Dice coefficients) between the positive and negative components of each task network and each of 17 canonical large-scale networks defined in the Gordon et al. parcellation28. Positive and negative components were analyzed separately because we observed visually that the positive and negative parts of each component typically represented distinct networks (e.g., for Component 5, the positive part involved the DMN, whereas the negative part involved the lateral visual network). More generally, the tensor ICA can identify states comprised of anti-correlated network patterns that have been reported by others during both resting state and task performance (see, for example, Li et al.52) as an important characteristic of brain organization. As such, the positive and negative parts were analyzed separately to determine their distinct network makeups. In order to assess statistical significance, a spin test implemented in Python was performed with 1000 permutations for each reference atlas network51. Significant overlap was thresholded at p < 0.05.

For the machine learning regression analyses, the number of trees in the random forest was set to 1000. No maximum depth was specified for trees; the minimum number of samples required at each leaf was set to 1, the minimum number of samples required to split an internal node was set to 2, and the maximum number of features to consider when looking for the best split was set to 1. This analysis was carried out using scikit-learn53.

To assess amygdala functional connectivity with each of the large-scale brain networks, we used the dual regression approach35. In the first stage, the full set of group-level tensor ICA spatial maps was regressed against each participant’s task fMRI data for each run. In the second stage, the resulting time courses from this regression were used in a second regression against the fMRI data to compute the subject-specific functional connectivity map for each network. The average regression weights in the amygdala regions of interest were then computed from each of the 10 network maps for each participant (for each run). These values were used for further analyses to assess relationships between non-imaging variables and amygdala connectivity.

Statistics and reproducibility

For reproducibility assessment, we randomly separated eligible subjects into two groups with non-overlapping family structure (Group 1: n = 413; Group 2: n = 416). We focused the main analyses and discussion on task-evoked brain networks and brain-behavior relationships that replicated across both groups. Given the oversampling of siblings and twins in the HCP-YA cohort, we consolidated subjects from the same family within the same group to prevent family structure-related data leakage across the groups. For within-group analyses, we used PALM to correct for covariance in family structure. Statistical analyses used a conservative Bonferroni correction for multiple comparisons.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.