Introduction

Emotional attention corresponds to the prioritization and preferential processing of negative or threat-related stimuli in the environment to yield fast and adaptive behavioral changes, such as flight or fight responses, in turn increasing survival1,2,3. Converging evidence from animal and human neuroscience shows that it is achieved and coordinated by a distributed brain network comprising limbic and ventral medial prefrontal areas, which can remotely influence visual areas of the occipital and temporal cortices, thereby amplifying the sensory processing of these threat-related stimuli4,5,6. Remarkably, this brain network appears to have evolved in parallel with other ones involved in the control of attention when neutral stimuli are selected and further processed. According to a dominant neurobiological model7, two dissociable fronto-parietal networks coexist to promote either a top-down or bottom-up processing of (neutral) visual stimuli and their reciprocal interactions eventually determine whether the selection occurs based either on goals (top-down attention) or physical salience (bottom-up attention). As a result of this architecture and brain mechanisms, at least three sources of information can in principle guide or bias the selection of information available in the (proximal) environment: threat, physical salience, and goal5,8,9,10. Moreover, if value is defined by reward rather than threat, a similar tripartite model of selective attention has been put forward11 and whether value, goal, or physical salience is eventually prioritized at a given time is actually determined by an overarching priority map held and computed in these fronto-parietal networks12. A direct implication of this theoretical framework is that attention control is flexible and determined by multiple cues or sources of information. Yet, it still remains unclear how this flexibility emerges and in particular, when threat and goal compete with one another for attention selection, how is this competition eventually resolved.

On the one hand, there is ample evidence from psychiatry and (internalizing) psychopathology that this competition can be strongly biased and lead to an inflexible balance between threat and goal, whereby the former one can readily impinge on the latter one and as a result, the processing of negative or threat-related stimuli is inadvertently amplified or exacerbated even though they are not goal relevant13,14. This in turn can lead to the etiology and maintenance of affective disorders, such as generalized anxiety or phobia15. On the other hand, in healthy adult participants, recent behavioral studies have reported that the automatic selection of threat can temporarily be downplayed or reduced when a concurrent goal is activated and pursued16,17,18,19, suggesting that in these participants, top-down attention control can create a bias away from threat (i.e. value). Hence, attention control is probably flexible and adaptive for them. But how does this change in emotional attention triggered by goal happen and what is the brain mechanism behind it?.

The goal of our study was to address this important question. To this end, we harnessed the high temporal resolution provided by electro-encephalography (EEG), focusing on specific visual event-related brain potentials (ERPs) recorded from the scalp in healthy adult participants and previously associated with emotional attention5. Moreover, because previous studies have documented a modulation of emotional attention by goal at the behavioral level using the DPT18,19,20, here we combined for the first time this specific paradigm with 64-channel EEG (as well as eye-tracking to ensure that central fixation was used by the participants) to examine at which processing stage(s) this modulatory effect of goal on emotional attention could take place.

In the DPT, on each and every trial, a pair of faces is first briefly presented, with one of them shown along the horizontal axis on the left side while the other one is shown on the symmetric right side relative to central fixation. Importantly, one of them is neutral while the other one is emotional, either fearful or happy. Shortly after this pair of faces, a unilateral target (i.e. a dot or another shape) appears at one of these two special locations, hence either on the left or right side compared to central fixation21. Participants are asked to process this target (either to detect or discriminate it) and even though the emotional face is never predictive of the target’s location (and moreover is never task-relevant), performance is however influenced by it. More specifically, performance improves if the target replaces the position previously occupied by the emotional face (i.e. valid trial) compared to the neutral face (i.e. invalid trial), especially if it is a fearful face 22,23,24. This result suggests an automatic spatial orienting of attention towards the emotional face, which is potentiated by fear. However and importantly, this emotional bias is not automatic because it can be modulated by goal25,26,27. This is usually demonstrated by interleaving induction trials with the dot probe trials18,19. These induction trials require participants to process the cue (e.g. to report the valence of the emotional face)20. When induction trials are combined with the dot probe trials, behavioral results show that fearful faces cease capturing attention during the DPT and this effect is interpreted as a pre-emption of goal (over threat) during attention control.

Here we adopted this modern approach and asked 40 participants to perform the DPT either alone in some blocks or in combination with induction trials in other ones, with the order of these two conditions (i.e. Goal vs. NoGoal) being alternated across them. Accordingly, we could compare for the exact same participants (i.e. within-subject design) emotional attention when the effect of goal relevance was low and high, respectively. Moreover, because we used EEG, we could extract and score specific ERP components either time-locked to the cue or the target to assess how goal could influence emotional attention. At the cue level, we focused on the face-specific N170 component28 for which mounting ERP evidence suggests that it is amplified by negative emotion, including fear29,30. Moreover, we also analyzed the EPN31, which follows the N170 and is sensitive to emotional attention (i.e. it reflects an early attention allocation process to the emotional stimulus)32,33,34. At the target level, we computed lateralized visual ERPs because we used a unilateral target35,36. We primarily focused on the posterior contralateral P1 (P1pc)37,38 since previous ERP studies using the DPT have found that emotional attention, very much like visuo-spatial attention with neutral stimuli during the Posner cueing task35, influenced the extrastriate P1 component, i.e. its amplitude was larger for valid than invalid trials during this task24. To reduce motor contamination, we implemented a go/no-go task, in which participants were asked to press a button in response to a specific deviant target on a minority of trials. Consequently, even though we analyzed and reported behavioral performance for completeness (using the signal detection theory), our main focus was on the visual ERPs introduced here above and recorded for no-go trials. When target discrimination was required on each and every trial (instead of go/nogo task) in this paradigm, the behavioral results showed that negative faces captured attention and this effect decreased with the inclusion of induction trials20.

Results

Behavioral results

The ANOVA for the d prime scores (see Fig. 1A) revealed a significant main effect of Goal (F1,39 = 8.214, p = 0.007, ηp2 = 0.174, BFincl = 6.097 suggesting moderate evidence), showing a lower sensitivity for Goal than NoGoal trials. The main effect of Emotion was marginally significant (F1,39 = 3.289, p = 0.077, ηp2 = 0.078, BFincl = 0.399 suggesting anecdotal evidence), with a higher sensitivity for happiness than fear. The main effect of Validity was non-significant (F1,39 = 0.120, p = 0.731). Moreover, the interaction between Emotion and Validity, Emotion and Goal, and Validity and Goal all were non-significant (all Fs < 1, ps > 0.5). The three-way interaction between Emotion, Validity, and Goal was also non-significant (F1,39 = 1.025, p = 0.318).

Fig. 1
figure 1

(A) Mean d prime values (± SE). (B) Response bias (± SE). (C) Mean RTs for the NoGoal and Goal conditions, separately. (D) Mean ACC for the induction trials. (E) Mean RTs for the induction trials (correct responses only). *** p< 0.001.

Full size image

For the response bias (see Fig. 1B), the main effects of Emotion (F1,39 = 4.342, p = 0.044, ηp2 = 0.100, BFincl = 0.695 suggesting anecdotal evidence) and Goal (F1,39 = 4.289, p = 0.045, ηp2 = 0.099, BFincl = 1.562 suggesting anecdotal evidence) were significant. Moreover, the interaction between Emotion and Goal was also significant (F1,39 = 5.834, p = 0.021, ηp2 = 0.130, BFincl = 3.336 suggesting moderate evidence). The post-hoc t-test showed a higher response bias for fearful than happy trials in the Goal condition (t39 = 3.190, p = 0.012, Cohen’s d = 0.335). This emotion effect was not significant in the NoGoal condition (t39 = −0.477, p = 1.000). The main effect of Validity (F1,39 = 1.738, p = 0.195) was not significant. Likewise, the two-way interaction between Validity and Goal (F1,39 = 0.350, p = 0.558), Emotion and Validity (F1,39 = 0.415, p = 0.523), and the three-way interaction between Emotion, Validity, and Goal (F1,39 = 1.446, p = 0.236) were all not significant.

Regarding the RTs (see Fig. 1C), the main effect of Goal was significant (F1,39 = 99.916, p < 0.001, ηp2 = 0.719, BFincl = 4.809*10+35 suggesting decisive evidence), showing faster target processing during the DPT for the NoGoal than Goal condition. All other effects remained non-significant: Emotion (F1,39 = 0.766, p = 0.387), the two-way interaction between Emotion and Validity (F1,39 = 6.517*10–4, p = 0.980), between Emotion and Goal (F1,39 = 2.301, p = 0.137), between Validity and Goal (F1,39 = 1.000, p = 0.324), and the three-way interaction between Emotion, Validity, and Goal (F1,39 = 0.052, p = 0.820).

For the induction trials (see Fig. 1D,E), the mean accuracy (ACC) was 78.9% (SD = 0.08) and the mean RT for correct responses was 1043 ms (SD = 170.80). The mean ACC for fearful-neutral and happy-neutral pairs was 84.4% (SD = 0.12) and 73.2% (SD = 0.11), respectively. They were statistically different from each other (t39 = 5.512, p < 0.001, Cohen’s d = 0.724). The mean RTs for fearful-neutral and happy-neutral pairs were 1059 ms (SD = 163.98) and 1066 ms (SD = 165.79), respectively. They did not differ from each other (t39 = −1.446, p = 0.156).

N170 at the cue level

The ANOVA showed a statistically significant main effect of Emotion (F1,39 = 8.744, p = 0.005, ηp2 = 0.183, BFincl = 3.732 suggesting moderate evidence; see Table 1 and Fig. 2), indicating a larger N170 for the fearful-neutral compared to happy-neutral face pairs. The effect of Goal (F1,39 = 1.057, p = 0.310) was not significant, nor was the interaction between these two factors (F1,39 = 0.072, p = 0.789).

Table 1 Mean amplitude in µV (std) for the N170 and EPN at the cue level.
Full size table
Fig. 2
figure 2

(A, B) Grand average EPRs for the face pairs (electrodes P8 and P10 are pooled together). (A) N170 for fearful (blue) and happy (red) faces, irrespective of Goal. (B) EPN for NoGoal (blue) and Goal (red) conditions, irrespective of Emotion. On the ERPs, the error bar corresponds to ± 1 standard error of the mean (SEM). (C, D) Mean ERP amplitudes for N170 (C, 165–185 ms) and EPN (D, 270–300 ms) across conditions. (E, F) The corresponding horizontal topographical scalp map for the N170 (E) and EPN (F) in each condition is shown. * p < 0.05; ** p < 0.01.

Full size image

EPN at the cue level

Following the N170 component, a second and sustained negative activity was recorded at the same lateral occipito-temporal electrodes (see Table 1 and Fig. 2), sharing similarities with the EPN31. The ANOVA run on the amplitude values of this EPN showed that the main effect of Goal was significant (F1,39 = 29.650, p < 0.001, ηp2 = 0.432, BFincl = 5170.327 suggesting decisive evidence), with larger amplitudes for Goal than NoGoal. The main effect of Emotion was marginally significant (F1,39 = 3.240, p = 0.080, ηp2 = 0.077, BFincl = 0.684 suggesting moderate evidence), with larger amplitudes for fear than happiness. Moreover, the interaction between Emotion and Goal was significant (F1,39 = 4.473, p = 0.041, ηp2 = 0.103, BFincl = 1.874 suggesting anecdotal evidence). Post-hoc t-tests showed that the amplitude of the EPN was larger for fear than happiness in the Goal condition (t39 = 2.742, p = 0.015, Cohen’s d = 0.144). However, this emotional effect was not significant in the NoGoal condition (t39 = 0.003, p = 0.997).

P1pc at the target level

The ANOVA for the P1pc showed a significant main effect of Validity (F1,39 = 19.268, p < 0.001, ηp2 = 0.331, BFincl = 285.928 suggesting decisive evidence), with a larger P1pc for valid than invalid trials. The interaction between Emotion and Validity was significant (F1,39 = 4.388, p = 0.043, ηp2 = 0.101, BFincl = 1.340 suggesting anecdotal evidence). Post-hoc t-tests showed that while the P1pc amplitude was moderately larger for fear-valid compared to fear-invalid (t39 = 2.280, p = 0.077, Cohen’s d = 0.325), it was definitely larger for happy-valid compared to and happy-invalid trials (t39 = 4.774, p < 0.001, Cohen’s d = 0.681). The interaction between Emotion and Goal was also significant (F1,39 = 5.117, p = 0.029, ηp2 = 0.116, BFincl = 1.479 suggesting anecdotal evidence), though none of the post-hoc tests reached significance. Importantly, the interaction between Emotion, Validity, and Goal was almost significant (F1,39 = 3.837, p = 0.057, ηp2 = 0.090, BFincl = 1.986 suggesting anecdotal evidence) suggesting that the spatial orienting towards emotional faces captured by this early visual component was influenced by Emotion and Goal. To corroborate this conclusion, we analyzed the effects of Emotion and Validity separately for the NoGoal and Goal conditions.

In the NoGoal condition, the ANOVA showed a significant main effect of Validity (F1,39 = 11.309, p = 0.002, ηp2 = 0.225, BFincl = 21.503 suggesting strong evidence; see Table 2 and Fig. 3), with a larger P1pc for valid than invalid trials. Post-hoc t-tests showed a larger P1pc for fear-valid than fear-invalid trials (t39 = 2.544, p = 0.053, Cohen’s d = 0.453) and a larger P1pc for happy-valid than happy-invalid trials (t39 = 3.037, p = 0.017, Cohen’s d = 0.540). Neither the main effect of Emotion (F1,39 = 1.750, p = 0.194) nor the interaction between these two factors (F1,39 = 0.195, p = 0.661) was significant.

Table 2 Mean amplitude in µV(std) for the P1pc at the target level.
Full size table
Fig. 3
figure 3

P1pc results for the NoGoal condition. (A, B, D, E) Grand average EPRs for the target at electrode PO7/PO8, for each condition separately. The blue line represents the electrode ipsilateral to the target while the red line represents the contralateral electrode. The dotted line represents the difference wave (contralateral minus ipsilateral). (C, F) Difference waves (contralateral minus ipsilateral) for fear (C) and happy (F) conditions, comparing valid (pink) and invalid (purple) trials. (G). The corresponding horizontal topographical scalp map for the P1 in each condition is shown (before the subtraction). (H). Mean P1pc amplitude in the 120–210 ms time window following target onset. On the ERPs, the error bar corresponds to ± 1 standard error of the mean (SEM). * p< 0.05; ** p < 0.01.

Full size image

In the Goal condition, the ANOVA showed a significant main effect of Validity (F1,39 = 11.791, p = 0.001, ηp2 = 0.232, BFincl = 21.066 suggesting strong evidence; see Table 2 and Fig. 4), with a larger P1pc for valid than invalid trials. The main effect of Emotion was marginally significant (F1,39 = 3.364, p = 0.074, ηp2 = 0.079, BFincl = 0.563 suggesting anecdotal evidence), with larger P1pc amplitudes for fear than happiness. Moreover and importantly, the interaction between Emotion and Validity was significant (F1,39 = 7.204, p = 0.011, ηp2 = 0.156, BFincl = 11.996 suggesting strong evidence). Post-hoc t-tests showed that the amplitude of the P1pc for fear-valid trials was not significantly larger than fear-invalid trials (t39 = 1.017, p = 0.625, Cohen’s d = 0.192), while happy-valid clearly elicited a larger P1pc than happy-invalid trials (t39 = 4.358, p < 0.001, Cohen’s d = 0.826).

Fig. 4
figure 4

P1pc results for the Goal condition. (A, B, D, E) Grand average EPRs for the target at electrode PO7/PO8, for each condition separately. The blue line represents the electrode ipsilateral to the target while the red line represents the contralateral electrode. The dotted line represents the difference wave (contralateral minus ipsilateral). (C, F) Difference waves (contralateral minus ipsilateral) for fear (C) and happy (F) conditions, comparing valid (pink) and invalid (purple) trials. (G). The corresponding horizontal topographical scalp map for the P1 in each condition is shown (before the subtraction). (H). Mean P1pc amplitude in the 120–210 ms time window following target onset. On the ERPs, the error bar corresponds to ± 1 standard error of the mean (SEM). *** p< 0.001.

Full size image

Correlation between EPN and P1pc

When computing a difference wave for the EPN (i.e. fear minus happiness ERP activity), we found that it correlated negatively with the P1pc for fear-invalid trials (r = −0.321, p = 0.044), happy-invalid trials in the NoGoal condition (r = −0.341, p = 0.031), and fear-invalid trials in the Goal condition (r = -0.321, p = 0.044) (see Fig. 5).

Fig. 5
figure 5

Scatterplot of the significant correlations found between the EPN and P1pc for fear-invalid trials of the NoGoal condition (left panel), happy-invalid trials of the NoGoal condition (middle panel), and fear-invalid trials of the Goal condition (right panel). For happy-valid trials of the Goal condition, this correlation was not significant (r = − 0.125, p = 0.443).

Full size image

Discussion

In this study, we investigated the modulatory effects of goal relevance on emotional attention and used 64-channel EEG to shed light on the brain mechanisms behind them. To this end, we employed a standard DPT with emotional faces (either fearful or happy faces) that had to be performed either without (NoGoal condition) or with induction trials (Goal condition), with the order of these two conditions alternated across participants. At the cue level, we extracted and scored the face-specific N170 component as well as the subsequent EPN, while at the target level, we focused on the P1pc, which is a lateralized visual ERP that was clearly elicited in our experiment.

At the cue level, we found that the face-specific N170 component was larger when the cue included a fearful compared to a happy face, and importantly, this early threat-related amplification was not modulated by goal relevance, suggesting that it was likely automatic, in the sense of goal-independent27. Since the N170 reflects the early structural encoding of faces28,39,40,41, our results suggest that if the face was threat-related (i.e. fearful), then this encoding was potentiated5,6. Previous ERP studies have already reported amplitude modulations of the N170 by fearful faces29,42,43,44 and moreover this amplification was found to be task-independent in some of them45,46,47. Our results for the N170 are compatible with them and they suggest that this early occipito-temporal ERP component captures a rather automatic extraction of emotional relevance, which is bolstered by fearful faces45. According to a dominant emotional attention model6, this face-specific amplification could be driven by the amygdala and acts as a fast-responding relevance detector48, which can rapidly bias, through distant anatomical projections, sensory processing in the occipital cortex irrespective of top-down attention control mechanisms49,50. However, whether or not the amygdala is causally involved in this early modulation of the N170 ERP component by fear is still debated in the existing literature51 and additional studies with a better spatial resolution than scalp EEG (or alternatively, multimodal studies) are needed to elucidate its actual role in emotional attention. For example, intracranial EEG (iEEG) could provide more direct evidence for the neural basis of this effect. Using a different task, Pourtois et al.52,53 previously found that early on following stimulus onset, the amygdala discriminated fearful from happy faces irrespective of attention, while in the ventral occipital cortex (where the N170 is generated), negative emotion influenced neural processing at a later stage, in agreement with a possible feedback or reentrant effect from the former area onto the latter one.

Although the N170 component was amplified by fear irrespective of goal, the subsequent EPN showed a different pattern and increased amplitude if and only if this fearful face was goal-relevant. Accordingly, compared to the N170, the EPN captured a different attention effect. More specifically, unlike the N170 whose modulation by fear appeared to be automatic, this effect for the EPN depended on the goal relevance of emotion, thereby suggesting not only dissociable neurophysiological effects for these two successive ERP components but also a likely hierarchization of information processing driven by the cue whereby threat processing could precede goal relevance29,42,45. Consistent with this notion, several previous ERP studies have already reported an amplitude modulation of the EPN by threat54,55,56, in particular when threat is goal or task-relevant42,47,56. Accordingly, one could imagine that evolutionary relevant stimuli such as fearful faces are rapidly processed irrespective of goal at the N170 level31,42, before goal relevance could further assist or boost their emotional differentiation at the EPN level54,57. More generally, these dissociable ERP results found for the N170 and EPN corroborate the notion of an attentional priority map, where value (threat) and goal can each contribute to stimulus processing and eventually bias target’s selection5,11,58.

Despite this prioritization of fear at the cue level, the P1pc results for the target clearly show that when emotion was not goal-relevant, the participants oriented preferentially to the side occupied by the emotional face, and this, irrespective of its actual valence. The P1pc was larger for valid than invalid trials, regardless of the valence of the emotional face, suggesting that emotion, as opposed to threat per se, actually guided target’s selection. More specifically, because the emotional face (either fearful or happy) was not predictive of target’s location (i.e. valid and invalid trials were equiprobable), these P1pc results suggest that target processing was enhanced when it was shown on the same side as the emotional (valid trials) compared to the neutral face (invalid trials), unequivocally translating an attentional bias for these emotional faces in this DPT. However and importantly, when emotion was goal-relevant, then we observed a substantial change in the way participants selected it, namely fear ceased to capture attention while happiness still did it. In this condition (Goal), the P1pc was not significantly larger for fear-valid than fear-invalid trials while happy valid trials still elicited a larger P1pc than happy invalid trials. This result unambiguously suggests that goal relevance can influence emotional attention early on following target onset at the P1 level in the extrastriate visual cortex where this visual ERP component is generated35,59. However, because the three-way interaction between Emotion, Validity, and Goal was marginally significant only, this modulation of emotional attention by goal relevance at the P1pc level appears to be modest in magnitude and hence some caution is needed in this interpretation. Yet, this lack of validity effect for fearful faces when emotion is goal-relevant stands out and suggests that participants likely stopped to orient to the side occupied by these negative emotional faces at the cue level, probably because spatial attention was no longer selective, but instead distributed across the two faces (i.e. fearful and neutral) that competed with each other for attention selection. In agreement with this interpretation, two behavioral results of our study are noteworthy. First, we found that in this condition, the response bias, but not sensitivity (reflected by d prime), increased at the behavioral level. According to the signal detection theory60, this change reflects the adoption of a more liberal response criterion. Translated to our paradigm, the participants therefore favored target (go) over non-target (nogo) when emotion was goal-relevant and a fearful face was used at the cue level, selectively61. Because this response bias was not modulated by validity, it suggests that spatial attention was probably distributed or diffuse when the target appeared after the cue (with a fearful face) and was shown either on the left or right side of the screen relative to fixation. The propensity of threat or fear to create response biases, especially under uncertain conditions, is not new but backed up by several previous studies and theoretical models available in the literature10,62,63. It can even be interpreted as an evolutionary and adaptive mechanism to some extent1,64: because threat is goal-relevant, lowering the decision threshold for target processing that accompanies these negative stimuli ensures that they are less likely to be overlooked, even at the cost of an increased rate of false alarms. Second, when examining behavioral performance for the induction trials, we found that participants were actually better at discriminating fear from happiness based on these cues composed each time of an emotional face and a neutral face. To account for this effect, it appears plausible to assume that participants perceived neutral faces as somewhat negative65,66,67 and hence their competition with fearful faces was enhanced during the induction trials compared to happy faces. As a result, spatial attention was probably less selective (i.e. less biased towards the side occupied by the emotional face) for fearful-neutral than happy-neutral pairs in the Goal condition. In other words, goal relevance likely exerts dissociable effects on the processing of fearful and happy faces, and if the competition with neutral faces is exacerbated (because neutral and fearful faces are both perceived as negative or threatening stimuli), then a stronger modulation of emotional attention could be detected (at the P1pc level) than when this competition is low (when a happy face competes with a neutral face for attentional selection). Hence, when we consider the P1pc results together with these behavioral effects, it appears plausible to posit that a change regarding the selectivity of spatial attention occurred as a function of goal relevance and threat.

To further corroborate this interpretation, we also explored whether the P1pc could be predicted by the preceding EPN at the cue level or not. We reasoned that if our goal manipulation did change the way spatial attention was allocated to the emotional face at the cue level, then the P1pc might be related to the EPN because this ERP component showed a significant interaction between Emotion and Goal at the cue level. As can be seen from Fig. 5, we did find a negative relationship between the EPN effect (when subtracting happy from fearful faces) and the P1pc, however, for invalid trials only. More specifically, across participants, the P1pc was larger when this EPN effect was stronger, except for happy valid trials in the Goal condition where this correlation was not found. These results indirectly suggest that the more the participants prioritized the neutral face at the cue level, the larger their P1pc was for a target appearing shortly after at the position occupied by it. Conversely, these results suggest that if participants prioritized the emotional face, then their P1pc for an invalid trial was reduced. Although it remains currently unclear why similar correlation results between the EPN and P1pc were not found for valid trials, they nevertheless inform about possible inter-individual differences in emotional attention and enable us to link more directly the spatial orienting effect occurring at the target level (P1pc) with a preceding and proximal event thought to causally influence it, namely the cue (EPN).

Because the DPT has been used and adapted in the past to treat attentional biases to threat in specific affective disorders such as generalized anxiety disorder (GAD) or depression (i.e. attention bias modification—ABM)68, we believe our new ERP results might also be relevant to them to some degree69,70,71. In those studies, the participants are usually trained to attend preferentially to the side occupied by the neutral (i.e. safe) face during the DPT because it predicts target’s location with a high probability, much higher than the fearful or threatening face. This way, a specific association between neutral faces and target’s location can be created, learned, and used to steer attentional control. After training, one can measure attentional biases towards threat and if the procedure is successful (but see MacLeod et al.72 and Mogoaşe et al.73 for mixed results), then they are reduced compared to the pre-training measurement or phase. In comparison, here we show using a different experimental procedure based on the use of interleaved induction trials and devoid of training18,19 that the spatial orienting to emotional faces could be altered by them (with clear neurophysiological effects found for both the cue—EPN and the target—P1pc) and as such, it might provide an efficient means to reduce (and measure with ERPs) attentional biases to threat in specific affective disorders in the future. As a caveat, we note however that in our study, induction trials did not produce a simple bias away from threat, but instead they altered the automatic spatial orienting to fearful faces because the competition with neutral faces was temporarily enhanced in this condition (Goal). Hence, it remains to be shown whether this dynamic change of spatial attention triggered by the induction trials could reduce attentional biases in patients who are characterized by an enhanced vulnerability to fear.

Because previous studies on emotional attention and goal relevance have mostly used behavioral measures, such as RTs and ACC18,19,20,74, our new ERP results extend them by showing that the modulation of the former process by the later one can be traced early on after target onset in the visual cortex, namely at the P1pc level. Hence, an early/perceptual locus for it can be postulated, as opposed to a late/post-perceptual one. In other words, goal relevance appears to exert a rapid top-down modulation of sensory processing in the visual cortex in conjunction with value. Accordingly, emotional attention is not automatic, but flexible and modulated by goal relevance. In addition, since we have used eye-tracking and removed trials contaminated by eye movements, this early P1pc effect cannot be attributed to them and hence it likely reflects the involvement of covert attention mechanisms. Likewise, because we mostly focused on and analyzed no-go trials at the ERP level, the contribution of motor activity or preparation to this P1pc effect appears unlikely.

Limitations

A few limitations warrant comment. First, although we have used a previously validated procedure to manipulate goal relevance by means of induction trials in specific blocks18,19, these trials necessarily added a cognitive load (to the DPT), which might eventually create a slight imbalance in terms of working memory when comparing the Goal to the NoGoal condition75. To overcome this problem, a refined experimental design could be used in future studies where the goal would be manipulated at the single trial level (as opposed to interspersed induction trials used in specific blocks, such as done in our study). For example, a specific cue could inform participants at the beginning of each trial about which information to focus on during it (i.e., either the target only, as usually done with the DPT, or both the cue and the target). As such, goal relevance and emotional attention could probably be better integrated with each other at the single trial level. Second, in accordance with many previous studies23,24,65,76, we have used the Ekman faces77, which are somewhat “artificial” and unnatural even to some extent. Therefore, it remains to be shown whether similar behavioral and ERP effects could be found and replicated when other emotional (face) stimuli are used. Third, although we created a large number of face pairs at the cue level and minimized their repetition, the reported N170 emotional effect might be contaminated by this factor or habituation. In a control analysis (see supplementary materials, Figs. S1, S2), we addressed this possibility directly and found that this emotional effect at the N170 level decreased in magnitude (and was no longer significant) when stimulus repetition was included as a factor at the statistical level. Although this modulatory effect of stimulus repetition on the N170 goes beyond the scope of the present study, we believe that additional ones are needed to disentangle the effects of repetition vs. emotion on this face specific ERP component. However, the effect of Goal on the subsequent EPN component was still highly significant in this control analysis, thereby confirming that these two successive ERP components likely reflect different stages of emotional processing. Last and relatedly, since we used a standard ERP averaging procedure, we mostly focused on effects of emotional attention and goal relevance that were visible on grand average ERP waveforms, but the use of single-trial data analyses could provide additional insights into the complex relationship between cue- and target-related ERP activity, as assumed with the DPT. Yet, when using lateralized ERP components such as the P1pc, this might be challenging because the signal to noise ratio is probably low for it at the single trial level. Accordingly, refined experimental designs based on the DPT would probably be required in the future to examine the relationship between cue and target-locked ERP effects in this task, as well as further characterize its dynamic modulation depending on emotional attention and goal relevance concurrently.

Conclusions

To summarize, our new findings suggest that emotional attention is not automatic. When emotion is made goal relevant by means of induction trials, our ERP results show that although the processing of fearful faces is enhanced automatically at the cue level (N170 component), this manipulation substantially influences a later stage of processing (EPN component), and moreover, the participants do not orient preferentially towards the side occupied by them, as revealed by the target-locked P1pc component. In this condition, we observed a striking change in spatial attention, which turned out to become less selective and more distributed. Our behavioral results are also compatible with this interpretation. More generally, because we have used a within-subject design and we could compare emotional attention for the exact same face stimuli with or without goal relevance, our results show that attention control is flexible and driven by both value and goal. Remarkably, because different ERP components appear to be sensitive to them, this neurophysiological technique holds many promises to elucidate in a near future the neural architecture underlying the dynamic interplay of value with goal during selective attention.

Materials and methods

Study approval

All participants provided written informed consent and were compensated 25 euros for their participation. The study was approved by the local ethics committee of the faculty of psychology and educational sciences at Ghent University (file number: #2022-029). The experiment conformed to the ethical standards prescribed by the 1964 Declaration of Helsinki and subsequent revisions. Participants were given comprehensive instructions about the procedures and tests and confirmed their agreement both verbally and in writing. Informed consent was obtained from all participants.

Participants

We used the same sample size as in our previous behavioral study where we found a modulation of emotional attention by goal relevance20. Forty-two healthy adults were recruited via SONA, which is an online system at Ghent University. They all were right-handed and had normal or corrected-to-normal vision, no history of neurological or psychological impairment, and no current medication. One subject had low accuracy for the induction trials (i.e. falling 3 SDs below the mean). Another one had a large amount of omissions during the DPT task in the Goal condition. We removed these two subjects. Thus, the final sample consisted of 40 subjects (aged 18–31, mean age = 21.48 years, SD = 3.44 years, 8 males).

Apparatus

Participants were seated approximately 70 cm away from a 19-inch CRT screen with 1024 × 768 resolution (60 Hz), with their heads restrained by a chinrest in a soundproof experimental room. Stimulus presentation and response recording were controlled by E-prime (Version 3.0). For the responses, a response pad was used. The position of the left eye was monitored continuously using an Eyelink 1000 + eye-tracking system (SR Research) at a sampling rate of 1000 Hz. We used specific E-Prime Extensions for Eyelink to synchronize stimulus presentation with this eye-tracking device. EEG data were recorded continuously using a 64-channel BioSemi Active Two system (BioSemi, Amsterdam, The Netherlands). The sampling rate was 512 Hz and the CMS-DRL electrodes were used as an online reference. The electrooculogram (EOG) was recorded from four external electrodes placed on the outer canthi (to monitor horizontal eye movements) and above and below the left eye (to monitor eye blinks). Two other electrodes were placed on the left and right mastoids, which were used offline for re-reference.

Stimuli

We used the emotional faces from the Ekman dataset77 and selected 10 distinct identities (5 males and 5 females). For each of them, fearful, happy, and neutral facial expressions were selected, resulting in a set of 30 different face stimuli. Each face was trimmed to remove the hair, ears, neck, and non-facial information by an oval shape measuring 6 × 8 cm and was converted to grayscale. Each face stimulus was adjusted in ImageJ. A non-parametric analysis (based on a Kruskal–Wallis test) showed that the mean luminance and contrast were not statistically different between the three emotion categories [luminance: H (2) = 2.821, p = 0.244; contrast: H (2) = 1.506, p = 0.471].

While we used 30 different face stimuli, we eventually created 160 face pairs that were used at the cue level during the DPT (i.e. each pair always showed two faces with the same gender, but always from two different persons/identities). This way, we could strongly limit the number of repetitions at this level. One face in the pair had an emotional expression (either fearful or happy) while the other one was neutral. Each face was positioned 8 cm away from the fixation cross (1 × 1 cm) along the horizontal axis, with one of them on the left side and the other one on the right side. By systematically varying the emotional expression (fearful or happy) and the spatial location (left or right) of the emotional face within each pair, we created four distinct pair types: fearful-neutral, neutral-fearful, happy-neutral, and neutral-happy. For each combination, 40 different pairs were created.

After the cue, a unilateral target measuring 3 × 3 cm was presented. It corresponded to a square (dark grey color with hex code #131,313) that could be titled 45 degrees clockwise and become a diamond. On each side of the screen, two (white) placeholders were presented along with this (unilateral) target. They were used to increase target processing at the places where the two faces were previously shown. These placeholders corresponded to square brackets and their size was 6 × 8 cm, which was the same as the faces presented at the cue level. Because the background was black (and uniform) and the target was grey, the latter had low contrast, and hence spatial attention had to be oriented (covertly) to its location in order to process its shape (either square or diamond).

Procedure

The experiment consisted of two sessions separated by a brief pause, with their order counterbalanced across participants. In one session (i.e. NoGoal), only the DPT had to be performed and hence emotion was not goal-relevant (see Fig. 6A). In the other one (i.e. Goal), besides the DPT which was the main task, participants had to discriminate now and then (i.e. in 20% of the trials) the valence of the emotional face shown at the cue level; hence decide whether it was a fearful or happy face (see Fig. 6B). These induction trials that were interspersed with the dot probe trials and they served to render the emotion goal-relevant. All stimuli were shown on a black background. Each part included a practice block of 20 trials, followed by 4 experimental blocks of 80 trials. In the Goal condition, each block included 64 dot probe trials plus 16 induction trials. In order to minimize the contamination of the visual ERP data by frequent motor responses, we used a Go/No-Go task, as was previously used and validated24. Participants were instructed to press a predefined button of the response box when either the square (half of the participants) or the diamond (the other half) was presented but to withhold responding when the concurrent shape was presented. The Go/noGo ratio was 1:9.

Fig. 6
figure 6

(A) Structure of a dot probe trial (here fear valid). (B) Structure of an induction trial (here the emotional face, happy face, is shown on the right side). (C) Examples of the four main conditions used during the DPT.

Full size image

Each trial began with a fixation cross shown for 500 ms. After that, the cue, consisting of a pair of faces, was shown for 100 ms followed by a short and variable interval (100, 150, 200, 250, or 300 ms). Then the target, either a diamond or a square, was presented either on the left or right side of the screen for 150 ms, and at one of the two positions occupied by the two faces serving as cue. These specific parameters were previously used24 and they promote a rapid orienting of spatial attention towards the side of the emotional face (cue). They also allowed us to record reliable visual ERPs to this event as well as the subsequent target. A trial was coded as valid if the target replaced the position of the emotional face (either a fearful or a happy face) and invalid if it replaced the neutral face. However, emotion was not predictive of target location as it appeared with an equal probability at the location occupied by the neutral or the emotional face. Accordingly, there were four main conditions (see Fig. 6C): fear-valid, fear-invalid, happy-valid, and happy-invalid, with an equal number of valid and invalid trials. Participants were asked to press the key using their right index only when a specific shape was presented (see here above, Go/noGo task). For the induction trials, the participants had to discriminate the valence of the emotional face, either fear or happiness, shown at the cue level. To this end, they used two other buttons of the response box, whose mappings were counterbalanced across them.

After the EEG experiment, participants were asked to complete the trait version of the State-Trait Anxiety Inventory (STAI-T)78. We used the twenty-item version (Form Y-2) that assessed trait anxiety. All items were rated on a 4-point scale (e.g., from “Almost Never” to “Almost Always”). Higher scores on the inventory indicated higher levels of anxiety. The average score of the STAI was 42.25 (range 22–63; SD: 9.553) and fell within the normal range for trait anxiety in the general population79,80.

Behavioral data

We used MATLAB R2023a (The Mathworks Inc., Natick, MA, USA) to process the data and JASP 0.17 to analyze them at the statistical level. First, we removed the first trial of each block. Moreover, to ensure that participants processed all stimuli with peripheral vision, we removed offline the trials where the eye deviated more than 3 degrees away from the central fixation cross at both cue and target levels (see Tables 3, 4). To this aim, we created a region of interest (ROI) around it, where the border was delimited by a circle (X values: 417.28–606.72 pixels; Y values: 289.28 to 478.72 pixels), whose diameter was 3.7 cm at a viewing distance of 70 cm. After that, d prime, response bias c, and mean reaction time (RT) for correct responses were computed for each condition and each participant. We adjusted the extreme value of rates (i.e., 0 or 1) with 0.01 or 0.99. For each dependent variable mentioned here above, a three-way repeated-measures ANOVA with Emotion, Validity, and Goal as within-subject factors was run. The partial eta square (ηp2) values were also reported as an estimate of effect size. A Holm correction was used for post-hoc comparisons because it controls the family-wise error rate while maintaining higher statistical power compared to the Bonferroni correction. This method sequentially adjusts p-values in a less conservative manner, reducing the risk of Type II errors while still controlling for false positives.

Table 3 The average number of trials/epochs kept per condition for the cue.
Full size table
Table 4 The average number of trials/epochs kept per condition for the target.
Full size table

EEG preprocessing

EEG data pre-processing was implemented with the EEGLAB toolbox running on MATLAB R2023a (The Mathworks Inc., Natick, MA, USA). First, the data were filtered with a 0.1 Hz high-pass and a 40 Hz low-pass finite response filter with default settings and referenced to the average of the left and right mastoids. Then the EEG data was synchronized with the behavioral data (which had already been pre-processed based on the eye-tracking data to remove eye movements). This way, we could retain EEG trials without eye movements and for correct responses at the behavioral level only. The eye-tracking data was also synchronized with the EEG data using the EYE-EEG toolbox. Next, we extracted two epochs of interest for the trials devoid of motor responses (i.e. Nogo trials). The first one was -110/ + 300 ms around the onset of the cue. The second one was -110/ + 700 ms around the onset of the target. Then, we used the FASTER plugin to identify artifacts along five dimensions: (1) Channels whose mean correlation coefficient, variance, and Hurst exponent exceeded Z = ± 3 were removed and data at these electrodes were interpolated; (2) Epochs in which the amplitude range, the deviation from the channel average and the variance exceeded Z = ± 3 were rejected; (3) Some small artifacts might only exist on single channels within single epochs. Therefore, within each epoch, channels whose variance, median slope, amplitude range, and deviation from the channel average exceeded Z = ± 3 were interpolated using spherical splines; (4) A grand average dataset was then established. For each subject, if the amplitude range, variance, deviation from the channel average, and the maximum absolute value of the EOG channels exceeded Z = ± 3 of the grand average, the subject’s data were entirely removed; (5) Epochs with more than 12 interpolated electrodes were directly removed. In addition, we conducted Kurtosis analysis to remove epochs with abnormal peak values (Z > 5) and performed Spectra Estimate to reject epochs with muscle activity (i.e. the trial spectra deviated by + 25 or −100 dB in the 20–40 Hz frequency window). Further, independent components (ICs) analyses were performed and ICs related to eye, muscle, and cardiac artifacts were detected, and manually rejected based on visual inspection of the EEG and eye-tracking data. After these steps, we re-epoched the target-locked ERP data using a −110/ + 500 ms window around target onset to optimize their analysis. Each epoch was baseline-corrected using the 100 ms pre-stimulus onset. Tables 3, 4 show the average number of epochs kept for each condition separately for the cue and the target.

ERPs

At the cue level, we computed two ERP components, namely the face-specific N170 and the EPN. We also analyzed the P1 component (see supplementary materials). The N170 reflects the structural encoding of faces and reaches a maximum (negative) amplitude 150–200 ms after face onset at lateral occipito-temporal electrodes, especially in the right hemisphere28,81. Consistent with these electrophysiological properties, the N170 peaked at 175 ms following the cue at electrodes P8 and P10 in our study. Accordingly, for each subject and condition, we computed the amplitude of the N170 in a −10/ + 10 ms interval around this peak. For the EPN component, we extracted its amplitude during a 30 ms interval spanning from 270 to 300 ms (corresponding to the end of the epoch) after cue onset at the same electrodes (i.e. P8 and P10). We selected a 30 ms time window for the EPN based on previous ERP studies31,34,81,82 as well as a visual inspection of our ERP data. For each of these two cue-locked ERP components, the corresponding mean amplitudes were submitted to a repeated-measures ANOVA with Emotion and Goal as within-subject factors.

At the target level, we computed lateralized ERP components and focused on the P1pc37,38 in keeping with previous EEG experiments that consistently reported amplitude modulations of spatial attention at the P1 level59,83, including for emotional attention5,24,84. Using PO8 and PO7, we removed the ipsilateral activity (relative to the target position) from the contralateral one. More specifically, the lateralized response was calculated by subtracting the ipsilateral activity (PO7) from the contralateral activity (PO8) for the left target, and symmetrically for the right target (i.e. ipsilateral activity at PO8 minus contralateral activity at PO7). Then these two difference waves were averaged. As can be seen from Figs. 3 and 4, a clear positive component was visible on these difference waves, starting early following target onset during the P1 time interval and extending to the subsequent N1 component. The P1pc was measured as a mean amplitude in the 120–210 ms (NoGoal) or 120–240 ms (Goal) time interval following target onset. To assess whether the attentional bias varied depending on Goal and Emotion, we first performed an omnibus repeated-measures ANOVA with Emotion, Validity, and Goal as within-subject factors. Next, we examined the effects of Emotion and Validity separately for the Goal and NoGoal conditions. A Holm correction for multiple comparisons was applied to analyze the ERP data, as it was the case for the behavioral data. We also performed Peason’s correlations between the EPN at the cue level and the P1pc at the target level.

For all behavioral and ERPs analyses, the significance level was set to p < 0.05 and complementary Bayesian repeated-measures ANOVAs were also conducted using JASP (version 0.17), enabling us to quantify the amount of evidence for including or excluding the main effect and/or interaction under consideration.

Eye-tracking fixation maps

We used the iMap Toolbox to visualize and analyze eye fixations separately for the cue and the target85. For each of them, a linear mixed model (LMM) was used. The fixation durations were projected into the two-dimensional space according to their X and Y coordinates at the single-trial level. We smoothed the fixation duration map by convoluting it with a two-dimensional Gaussian Kernel function at 1° of visual angle. We selected the estimated method and rescaled the fixation map to 205*154 pixels (scale parameter: 0.2) to reduce computational time. We used a full model on the fixation duration map without normalization for the cue (PixelIntensity ~ Cue + Experiment + Cue: Experiment + (1|Subject)) and the target (PixelIntensity ~ Emotion + Validity + Goal + Validity * Experiment + Emotion * Goal + Emotion * Validity + Emotion * Validity * Goal + (1|Sub)). The resulting heat maps were computed to ascertain that participants had a similar pattern of eye fixations between conditions, both during the cue and the target. The results of these analyses (see supplementary materials; Figs. S6, S7) clearly confirmed that for the ERP data reported and analyzed, participants processed the cue and target with peripheral vision and kept central fixation. Importantly, no significant differences were found between conditions at the cue or target level, suggesting that eye fixations (for the trials kept after removing deviations, see Tables 3, 4) were comparable for them.