Cortical evidence accumulation for visual perception occurs irrespective of reports

Abstract

Perception is a multi-faceted, dynamical process that can be tackled empirically through measures of stimulus detectability and confidence. We recorded stereo-electroencephalographic data of 29 participants partaking in three pre-registered experiments to assess if evidence accumulation, a form of sequential sampling of sensory evidence, can explain perception. In an immediate-response experiment, high-gamma activity from individual channels and decoded multivariate latent variables in the visual, inferior frontal, and anterior insular cortices display a correlation between the slope of their increase and reaction times. In two further experiments, this signal in the ventral visual cortex differentiates between (1) stimuli reported as seen vs. unseen in delayed detection, (2) high and low intensity stimuli during passive viewing, and (3) levels of confidence when stimuli were detected. A computational model of leaky evidence accumulation can successfully reproduce both behavioral and neural data. Overall, we show that evidence accumulation explains subjective aspects of visual perception.

Introduction

Subjective perceptions is a dynamic phenomenon, with percepts continuously coming into our stream of experience¹. One conventional way to study the brain processes underlying such stimulus detection is to use degraded stimuli at perceptual threshold and contrast trials for which participants report having detected the stimulus with trials for which no detection occurred^{2,3,4,5,6,7,8,9,10}. Stimulus detection is not synonymous with sensory encoding, as the latter can be observed even when participants deny perceiving the stimulus¹¹. To disentangle stimulus detection from processes involved in explicitly reporting detection, researchers have developed paradigms to study perception independent of task demands^12,13,14. Additionally, percepts come with varying levels of confidence, which some hold to be an integral part of perceptual processing^15,16,17,18 and can be measured with confidence ratings.

What processes in the brain determine when stimuli are detected and the associated confidence levels? Evidence accumulation models posit that the brain accumulates noisy sensory signals over time to form a perceptual decision^19,20. Pioneering work on the neural basis of evidence accumulation focused on the role of parietal and frontal cortices in discrimination tasks^20,21,22,23. It is now clear (1) that neurons also display temporal dynamics consistent with evidence accumulation for detection^24,25,26 and (2) that evidence accumulation is widely distributed in the brain, spanning occipital, inferior temporal, parietal and inferior frontal cortices^{25,27,28,29,30,31,32,33,34,35,36,37} (see ref. ³⁸ for a review of the role of distributed cortical processing in decision making). Some of these accumulators might play a larger role in cognition, e.g., shaping percepts even in the absence of task-related processing^{17,26,38,39,40,41,42}. Additionally, prior research indicates that confidence results from a readout of accumulated evidence^{26,43,44,45,46}.

In a preregistered study (https://osf.io/8qe3a/), we used stereotactic electroencephalography (sEEG) to identify neural accumulators leading to the immediate detection of weak visual stimuli. We investigated if accumulators reflected stimulus perception when reporting was delayed or even absent, as previous studies highlighted the importance of disentangling activity involved in the process of reporting from that involved in perceiving per se^5,12,13,47. Beyond stimulus detection, we identified which brain regions encoded the confidence in having detected the stimulus. Specifically, we hypothesized that confidence judgments resulted from the difference between a detection threshold and the maximal level of accumulated evidence¹⁷. In a nutshell, we found evidence for the existence of a neural code in the ventral visual cortex that (1) reflects evidence accumulation; (2) persists when the report was delayed or absent; and (3) predicts participants’ confidence in seeing something.

Results

We analyzed sEEG activity from 3301 recording sites in 29 human participants with drug-resistant epilepsy. In all experiments, face images were embedded for three successive frames (600 ms) at a pseudo-random point within a stream of 13 phase-scrambled images (Fig. 1A). Face intensity was modulated to approach the participants’ detection threshold. To test whether evidence accumulation processes underlie stimulus detection and confidence, we asked participants to perform three separate experiments that required different behavioral responses. We analyzed univariate and multivariate brain responses as a function of face detection, reaction times, face intensity and confidence judgments. Specifically, we looked at high gamma activity (HGA), a proxy for local neuronal firing^{48,49,50,51,52,53}, in four pre-registered brain regions of interest (ROIs): the ventral visual cortex (VVC), the superior parietal cortex (SPC), the inferior frontal cortex (IFC), and the dorsolateral prefrontal cortex (DLPFC; Fig. 1D). The selected ROIs have been found to reflect evidence accumulation^{20,23,31,33,36} and play a role in stimulus detection^26,54,55,56. Note that some intracranial electrodes located in the ROI that we pre-registered as the IFC in fact had their deepest channels in the anterior insular cortex. For simplicity, we will refer to the IFC as a ROI, but we also report single channels located in the anterior insula.

Fig. 1: Experiment 1.

A Experimental procedure. Participants fixated on a cross superimposed on scrambled frame images. An embedded face stimulus was presented for three consecutive 200 ms frames at a random time starting 600 ms to 1800 ms after trial onset. Faces are replaced by a white placeholder in the figure to satisfy Nature’s policy about AI-generated images. When participants pressed the button between 400 ms and 1500 ms after face stimulus onset, the trial was considered a hit (trials with response times <400 ms or >1500 ms were excluded, see C and methods). When participants did not press, the trial was considered a miss. B Average and individual detection rates for all participants (n = 30), both in catch trials (black) where no target was presented and in threshold trials (blue) where the target was presented at participants’ detection threshold. C Mean percentage of responses as a function of reaction times (100 ms bins) across participants (n = 30). Dashed vertical lines show the 400 ms and 1500 ms limits for considering the trial as a hit (blue bars). Early responses (black bars) were not analyzed. D Analyzed channels normalized to an MNI template, with channels in the ventral visual cortex (VVC, dark blue), superior parietal cortex (SPC, light blue), inferior frontal cortex (IFC, green) and dorsolateral prefrontal cortex (DLPFC, yellow) and in other brain regions (black). E Total number of channels in the four ROIs. The box with black contours shows the proportion of responsive channels, defined as channels showing a significant positive difference between hit and miss trials. The red line indicates the maximum number of responsive channels expected by chance (α = 0.05). F Cross-validated average area under the receiving operator characteristic (AUROC) for the hit vs. miss decoder in the four ROIs (bar plot: average; dots: individual participants). Red lines indicate group-level chance-level performance, as assessed by permuting labels across trials (VVC: n = 18; SPC: n = 5; IFC: n = 19; DLPFC: n = 9). Red dot surrounds indicate participants who displayed above chance-level performance after false discovery rate correction. All error bars indicate 95% confidence interval.

Neural correlates of evidence accumulation in the ventral visual and inferior frontal cortices

We started by establishing which brain channels and regions reflected immediate detection and evidence accumulation for such detection. In experiment 1, participants provided an immediate report - they were instructed to press a key with their right hand as soon as they detected a face. Trials were considered a hit if participants pressed in a window finishing 1500 ms after face onset (53.86 ± 3.99%; mean ± SD; Fig. 1B, C), and a miss if they did not press. We removed trials with premature responses (up to 400 ms after the face onset; 2.16 ± 2.32%). We identified 1002 responsive channels (30.08 ± 11.88% of channels across participants, p < 10⁻⁵, Cohen’s h = 0.72, 95% CI [28.79%, 31.92%]), defined as showing a response to detected faces (i.e., more HGA in hit than miss trials) during the second following face onset using nonparametric clustering analyses⁵⁷. Note that responsive channels’ activity is not likely to reflect unconscious stimulus encoding, given that such encoding would also be observed in miss trials. Responsive channels were found in significant numbers in the four ROIs (all ps < 10⁻⁵, Fig. 1E, see Supplementary Fig. 1 for other brain areas). The number of channels with more HGA in miss than hit trials was not above the level expected by chance, either across all channels (158 channels, 4.79%, p = 0.7, Cohen’s h = −0.01, 95% CI [4.06%, 5.51%]) or in ROIs (all ps > 0.28). Hierarchical mixed-effects regressions fitted by time point and corrected for false discovery rate indicated that the time courses for the effect of immediate detection differed qualitatively across ROIs (0.31 to 0.94 s after face onset in the VVC, 0.06 to 0.9 s in the SPC, 0.28 to 1 s in the IFC and 0.44 to 1 s in the DLPFC). To characterize the latent variable leading to immediate detection, we developed a time-resolved decoding algorithm that classifies a trial as a hit when the maximum value of the weighted sum of all channels in the region of interest reaches a threshold, similar to a decision variable within an evidence accumulation framework³⁴. The decoders could successively classify hit vs. miss trials in all four ROIs (Fig. 1F). Further multivariate analyses are focused on those participants for which decoding was successful (VVC: 4/18 participants; SPC: 5/5; IFC: 13/19; DLPFC: 4/9; Supplementary Table 3). To summarize, univariate and multivariate analyses indicated the implication of several brain areas during immediate stimulus detection in experiment 1, including our four pre-registered ROIs.

To specify the role of evidence accumulation in stimulus detection, we examined the relationship between neural activity and reaction times in hit trials. More precisely, we sought to identify two neural markers of evidence accumulation. First, we looked for a negative correlation between response times and the derivative of stimulus-locked HGA, underlying that faster responses are associated with a steeper slope of the accumulator signal following stimulus onset. Such an effect of reaction times on the slopes of HGA was observable in 18.46% of all responsive channels (185 channels, p < 10⁻⁵, Cohen’s h = 0.44, 95% CI [16.06%, 20.87%]; Fig. 2A shows examples of such channels). At the level of ROIs, there was a statistically significant effect of reaction times on the slope of HGA in the VVC (t(2427) = 4.72; p < 10⁻⁵, β = 0.99, 95% CI [0.58, 1.4], BF₁₀ > 100) and the IFC (t(5465) = 3.83; p = 1.3 × 10⁻⁴, β = 0.096, 95% CI [0.47, 1.46], BF₁₀ = 33.33; Fig. 2B), but not in the SPC (t(1464) = 1.86; p = 0.06, β = 0.52, 95% CI [−0.03, 1.08], BF₁₀ = 1.41) nor in the DLPFC (t(1810) = 1.38; p = 0.17, β = 0.27, 95% CI [−0.11, 0.65], BF₁₀ = 0.27; Supplementary Figs. 2A, 3 and 4A). Second, we investigated whether the aforementioned latent variable trained to discriminate hit vs. miss trials (see above) also reflected evidence accumulation. We tested if the timing at which the decoded latent variable crossed a detection threshold (set individually for each participant to balance hit and miss trial detection) would predict response times (Fig. 2C). A positive correlation between reaction times and the timing of threshold crossing by the latent variable was found in all ROIs: for all 4 participants that showed significant hit vs. miss decoding in the VVC, for 7 out of 13 in the IFC and for 2 participants out of 5 in the SPC and for one out of 4 the DLPFC (Fig. 2D, E; Supplementary Table 3). Overall, these findings provide converging evidence that the VCC and IFC instantiate evidence accumulation pertaining to stimulus detection.

Fig. 2: Evidence accumulation signals.

A Averaged HGA traces from single channels in the VVC (top) and anterior insula (bottom). HGA in miss trials (red) and in hit trials separated in three terciles based on reaction times (blue). The vertical dashed lines indicate the onset and offset times of the face stimuli. The black bars indicate significant effects of detection (cluster-based permutation tests). Note that the channel in the insula was localized by the atlas as part of the IFC. B Averaged HGA across all responsive channels in the VVC (top) and IFC (bottom). The black bars indicate main effects of detection as assessed with sample-by-sample hierarchical mixed-effects regressions (corrected for false discovery rate across time). Response-locked plots can be found in Supplementary Fig. 5. C Representation of the decoding algorithm, with example hit (blue) and miss (red) trials for 3 channels (“chan”). A trial-by-trial detection signal was obtained by applying the weights of the multivariate classifier at each time point (b = bias; w₁, w₂, …, w_n = weights). The weights were chosen to optimize the distance between the maximum values of the latent variables in hit vs. miss trials. Trials were classified as hit if the maximum values of the latent variable crossed a threshold (purple line), and as miss otherwise. D Latent variables decoded from the VVC (dark blue) of one participant (G5; left panel) and the IFC (green) of another participant (G21; right panel), averaged over all miss trials and separated in three terciles based on reaction times for hit trials. Insets: Trial-by-trial depiction of the relationship between reaction times and the predicted time of threshold crossing of the decoded latent variable, with Spearman correlations. E Average Spearman correlation between observed and decoded response times across participants (bar plot: average; dots: individual participants). Red lines indicate group-level chance-level performance assessed by permuting labels across trials (VVC: n = 4, dark blue; SPC: n = 5, light blue; IFC: n = 13, green; DLPFC: n = 4, yellow). Red dot surrounds indicate participants who displayed above chance-level performance after false discovery rate correction. All shaded areas represent 95% confidence intervals.

In experiment 1, we identified two ROIs that reflected evidence accumulation when participants were required to respond immediately after detecting the stimulus. It remained unclear whether these signals were associated with detection itself, or rather with motor preparation and execution. We addressed this question in two further experiments.

The perceptual decision signal decodes delayed detection and passive perception

Immediately after completing experiment 1, participants performed a delayed-report (experiment 2) and a no-report (experiment 3) version of the task on the same stimulus sequence (Fig. 3A). In experiment 2, signals related to detection in the absence of a concomitant motor action were obtained by asking participants to wait for the end of the trial (i.e., a minimum of 1300 ms after stimulus onset) to respond. Face stimuli were either presented at perceptual threshold (threshold trials; 47.34 ± 7.04% trials reported as seen) or at a higher intensity defined as 125% of the perceptual threshold (suprathreshold trials, 76.5 ± 10.27% trials reported as seen, significantly more than in threshold trials; t = 18.07, p < 10⁻⁵, β = 0.65, 95% CI [0.58, 0.72]; Fig. 3B and Supplementary Fig. 6). In 17% of trials, no face was presented (catch trials). Participants could report that they saw more than one face in cases where a non-veridical percept of a face occurred during the same trials as a veridical percept of the stimulus. To make this response option seem sensible to the participants, we included one trial in each block that actually contained two suprathreshold faces, but excluded these trials from analyses. In experiment 3, we tested whether signals from the previous experiments were still present in the absence of any task, during passive viewing of face stimuli at threshold and suprathreshold intensities. Interpreted concomitantly with experiment 2, this third experiment made it possible to distinguish such perceptual signals from signals reflecting processes involved in reporting one’s perception of a face, and from signals reflecting the encoding of stimulus intensity only.

Fig. 3: Experiments 2 and 3.

A Experimental procedures. Face stimuli were presented at threshold or suprathreshold intensities. In experiment 2, participants reported a detection response at the end of the trial. In experiment 3, participants passively viewed the visual stream. Faces are replaced by a white placeholder in the figure to satisfy Nature’s policy about AI-generated images. B Mean and individual false alarm rates in catch trials (black) and hit rates when faces were presented at threshold (light blue) and suprathreshold (dark blue) intensities in experiment 2 (n = 30). C Averaged HGA traces from the same channels as in Fig. 2A, as a function of stimulus detection (hits or miss, in blue and red) and intensity (threshold or suprathreshold, in light and dark color shades) in experiment 2. This channel in the anterior insula was pre-registered as part of our IFC ROI. D The averaged activity across all responsive channels in the VVC (left panel) and IFC (right panel) in experiment 2. E Averaged HGA traces from the same channels as in Figs. 2A and 3C, as a function of stimulus intensity (threshold or suprathreshold, in light and dark green) in experiment 3 (right panels). F Similarly to D, the averaged activity across all responsive channels in the VVC (left panel) and IFC (right panel) in experiment 3. In C and D, the black bars indicate significant main effects of detection; in E and F, they indicate significant main effects of stimulus intensity (cluster-based statistics for individual channels, hierarchical mixed-effects regressions for ROIs; corrected for false discovery rate across time). G Averaged decoded latent variable in hit (blue trace) and miss (red trace) trials in experiment 2 (left panel) and in threshold and suprathreshold trials in experiment 3 (right panel) in the VVC of participant G5. H Same as G in the IFC of participant G21. All error bars and shaded areas indicate 95% confidence intervals.

Over a third of responsive channels reflected detection in the absence of immediate motor action, with more HGA in hit than miss trials in experiment 2 (393 channels, 39.22%, p < 10^-5, Cohen’s h = 0.9, 95% CI [36.2%, 42.24%]). This number increased to over half when looking only at responsive channels that also reflected evidence accumulation in experiment 1 (57.3%; see examples in Fig. 3C). Hierarchical mixed-effects regressions fitted to the average value of HGA in the responsive time window (see Methods) with detection reports and stimulus intensity as predictors showed a main effect of detection in all ROIs (VVC: t(7953) = 5, p < 10⁻⁵, β = 0.03, 95% CI [0.018, 0.042], BF₁₀ = 100; SPC: t(3832) = 3.5, p = 4.73 × 10⁻⁴, β = 0.038, 95% CI [0.017, 0.06], BF₁₀ = 6.67; IFC: t(13722) = 4.53, p < 10⁻⁵, β = 0.023, 95% CI [0.013, 0.034], BF₁₀ = 50; DLPFC: t(4520) = 2.62, p = 0.0089, β = 0.016, 95% CI [0.004, 0.029], BF₁₀ = 3.7, Supplementary Figs. 2B, 4B and 7A; see also Supplementary Fig. 8 for a control analysis). HGA was also influenced by stimulus intensity in the VVC (t(7953) = 3.43, p = 6.15 × 10⁻⁴, β = 0.009, 95% CI [0.004, 0.015], BF₁₀ = 5.88), the SPC (t(3832) = 2.88, p = 0.004, β = 0.01, 95% CI [0.003, 0.016], BF₁₀ = 3.22), the IFC (t(13722) = 5.09, p < 10⁻⁵, β = 0.007, 95% CI [0.004, 0.01], BF₁₀ > 100), but this effect was not statistically significant in the DLPFC (t(4520) = 1.94, p = 0.053, β = 0.006, 95% CI [0, 0.012], BF₁₀ = 1.61; Fig. 2D and Supplementary Fig. 6C). There was a significant interaction between detection and stimulus intensity in the VVC (t(7953) = 2.76, p = 0.006, β = 0.005, 95% CI [0.002, 0.009], BF₁₀ = 12.5) and IFC (t(13722) = 2.39, p = 0.017, β = 0.003, 95% CI [0.001, 0.006], BF₁₀ = 4.55; Supplementary Fig. 6E), which was explained by stimulus intensity influencing HGA in hit trials only. Time-resolved hierarchical mixed-effects regressions indicated that the effects of detection in experiment 2 occurred for similar durations as immediate detection effects in experiment 1 (0.33 to 0.91 s after face onset in the VVC, 0.46 to 1 s in the SPC, 0.31 to 1 s in the IFG and 0.67 to 0.96 s in the DLPFC), and that the interaction effect started around that same time in the VVC (0.34 s) while it emerged later in the IFC (0.57 s). Thus, all ROIs showed an increase in activity when the face stimulus was detected, irrespective of stimulus intensity and also when the report was delayed.

These results from experiment 2 indicate that the detection effects found in the ROIs do not reflect an encoding of stimulus intensity or of an immediate action (although they could still reflect delayed motor planning^21,58,59). However, they do not arbitrate whether regions only played a role in report formation in the context of the task or in perceptual processes irrespective of report. We reasoned that stimulus-locked HGA activity reflecting motor preparation in the context of a task should not be observed under conditions of passive viewing in experiment 3. On the contrary, ROIs involved in perceptual processing independent of reports should have higher HGA in trials where faces were presented at suprathreshold (defined as 150% the perceptual threshold) than threshold intensity. This is because faces would have been perceived more often in suprathreshold trials even during passing viewing (i.e., a no-report condition). In experiment 3, we found this effect of stimulus intensity in 9.68% of all responsive channels (97 channels, p < 10^–5, Cohen’s h = 0.18, 95% CI [7.85%, 11.51%]), and in 22.64% of responsive channels that reflected evidence accumulation in experiment 1 and detection in experiment 2 (examples in Fig. 3E), but only in 4% of non-responsive channels (p = 0.99). Hierarchical mixed-effects regressions showed that HGA was influenced by stimulus intensity in the VVC (t(5750) = 3.27, p = 0.001, β = 0.011, 95% CI [0.004, 0.017], BF₁₀ = 4.55), but did not show this effect in other ROIs (Fig. 3F and Supplementary Figs. 2C, 4C and 9A). While Bayesian analyses yielded inconclusive results in the IFC (t(13720) = 1.83, p = 0.067, β = 0.004, 95% CI [0, 0.008], BF₁₀ = 0.6), they brought evidence in favor of the absence of an effect of intensity in the SPC (t(2296) = 0.66, p = 0.51, β = 0.003, 95% CI [−0.005, 0.01], BF₁₀ = 0.27) and DLPFC (t(4623) = −0.02, p = 0.98, β = 0, 95% CI [−0.01, 0.01], BF₁₀ = 0.27). Several channels showed responses in all experiments in the anterior insula (Fig. 4B), although an effect of stimulus intensity in experiment 3 was not observed at the level of the region. Interestingly, the only region to show an effect of stimulus intensity in exploratory whole brain analyses was the lateral visual cortex adjacent to the VVC, despite many regions showing an effect of detection in experiment 2 (Supplementary Figs. 7B and 9B). The effect of stimulus intensity in the VVC started earlier (0.25 to 0.61 s after face onset) than effects of detection in experiment 1 and 2. In summary, while many regions were sensitive to detection when a report was required, we only found statistical evidence for stimulus perception in the absence of overt reports in the ventral and lateral visual cortices.

Fig. 4: Channel locations.

A Projection of channels localized in the VVC and lateral visual cortex on a transverse plane of the MNI template brain at coordinate Z = −17 mm (see Supplementary Figs. 10–11 for other slides and localizations in native space). B Same projection for channels in the IFC and insula at Z = 9 mm. Channels within 2 mm of the displayed slice are represented. The responsive channels reflecting evidence accumulation in experiment 1 are circled in red, those showing a main effect of detection in experiment 2 are colored in blue and those showing this effect and an effect of stimulus intensity in experiment 3 are colored in purple. Non-sensitive channels are in black. Note that many channels that were localized in the IFC by our automatic pipeline are actually located in the anterior insular cortex. A control analysis following a visual inspection by a trained neurologist revealed similar results in the anterior insula and in our pre-registered IFC ROI (Supplementary Fig. 12).

We showed that activity in the VVC is consistent with evidence accumulation and persists when reporting is delayed or even absent. However, these results could be driven by different neural populations within the same brain region that accumulate evidence towards an immediate motor response in experiment 1, delayed-report in experiment 2, or passive viewing in experiment 3. We accordingly tested whether a common neural population reflecting evidence accumulation as shown in Fig. 2 was involved in all three experiments. Using the weights from the multivariate decoders trained to discriminate between hits and misses in experiment 1, we decoded delayed stimulus detection in experiment 2 in all regions (VVC: 4/4 participants showing this effect; SPC: 3/5 participants; IFC: 4/13 participants; DLPFC: 3/4 participants; Fig. 3G, H; Supplementary Table 3). We then used a similar approach to decode face intensity in experiment 3. Suprathreshold vs. threshold trials were used as labels under the premise that activity in suprathreshold trials would resemble activity in hit trials in the previous two experiments. Face perception could be decoded with high performance in the VVC (AUROC = 0.69) for one out of two participants who performed experiment 3 (for the remaining two participants we could not acquire recordings in this experiment due to clinical constraints; Fig. 3G). No significant decoding was found for any participant in the other ROIs (Fig. 3H; Supplementary Table 3). Interestingly, high decoding performance was observed in the VVC even though suprathreshold stimuli may have remained unperceived in some trials, and threshold stimuli were arguably perceived in about half the trials, possibly less due to inattention during passive viewing. In line with our univariate results, stimulus intensity could also be decoded for one participant in the lateral visual cortex (Supplementary Table 4). Considering all the above results, the same latent variable in the VVC reflected immediate detection and evidence accumulation in experiment 1, delayed detection in experiment 2, and stimulus intensity during passive viewing in experiment 3. Interestingly, the shared response across experiments identified in the VVC appeared to be driven by channels located at a similar location in the middle to anterior fusiform gyrus (Fig. 4A and Supplementary Fig. 10).

Until now, we focused on perceptual contents that were driven by external sensory stimuli. In pre-registered analyses, we next assessed whether the latent variable reflecting stimulus detection in the VVC could also reflect non-veridical percepts, in trials where participants reported seeing a face when none was presented on the screen. As we had no external event to time-lock our analysis to, we examined latent variables in the entire trial window. Doing so, we found that decoded latent variables crossed a predefined threshold (see Methods) more often in false alarm trials (17.19 ± 15.86% of trials with no face) than in correct rejection trials (p = 0.034, permutation test; Fig. 5A) in experiment 2. Moreover, we found that this threshold was also crossed more often when participants reported seeing more than one face (6.87 ± 7.91% of trials where one face was presented) vs. only one face (p = 0.023, permutation test; Fig. 5B). Neither of those effects was found in any of the other three ROIs (Supplementary Fig. 13) or in the lateral visual cortex. These results, combined with the results of passive viewing in experiment 3, indicate that the neural code identified in the VVC reflects both veridical and non-veridical percepts.

Fig. 5: Non-veridical percepts in the VVC in experiment 2.

A The weights from the decoding algorithm represented on Fig. 2C were applied to the entire trial window in which no face was presented. The traces are examples of the decoded latent variable for false alarm (black) and correct rejection (green) trials (a. u. = arbitrary units). We measured the number of times that the variables crossed a threshold, set individually for each participant to balance hit and miss trial detection. B Participant-level and overall averages of the number of times that the decoded latent variable crossed the detection threshold, for trials where no face was presented and the participant reported seeing no face (correct rejection; green) vs. one face (false alarm; black; p = 0.034; n = 4). C The same for trials where one face stimulus was presented and the participant reported seeing no face (miss; red) vs. one face (hit; light blue; p < 10⁻⁵) and one face vs. more than one face (hit +; dark blue; p = 0.023; n = 4). In B and C, the results are for participants that showed significant decoding of hit vs. miss trials in the stimulus onset-locked window in experiment 1. Missing values correspond to participants without trials in one response category. P-values were obtained using an uncorrected, non-parametric permutation test (one-tail), where permutations were performed on the relevant trials for each participant and then comparing the participants’ average difference between the two conditions of interest were compared to the permuted distribution of this difference: ***p < 0.001, **p < 0.01, *p < 0.05. Error bars indicate 95% confidence intervals.

The results from experiments 2 and 3 indicate that HGA in the VVC played a role in perceptual processing and not in motor execution or planning. Another indication regarding this was obtained from electrical stimulations of bipolar contact pairs performed in the context of participants’ clinical screening. None of the stimulations performed on the VVC channels for the 4 participants for whom we could successfully decode detection reports lead to any motor action or motor-related sensation. We note, however, that this is a null finding, and that more elaborate stimulation procedures might be required to engage the motor system.

In this section, we have shown that the same neural code reflected evidence accumulation and detection irrespective of report. In the next section, we ask whether perceptual confidence also relies on this same neural code.

Functional markers of evidence accumulation and stimulus detection in the VVC also reflect confidence

In experiment 2, participants were probed for their confidence in their detection response (Fig. 6A). A hierarchical mixed-effects regression on confidence revealed significant main effects of stimulus intensity (t = 2.82, p = 0.005, β = 0.15, 95% CI [0.05, 0.25]) and detection (t = 2.53, p = 0.01, β = 0.11, 95% CI [0.03, 0.2]) as well as a significant interaction between stimulus intensity and detection (t = 4.88, p < 10⁻⁵, β = 0.21, 95% CI [0.13, 0.3]). To explore this interaction further, we analyzed hit and miss trials separately, and found that confidence increased with stimulus intensity for hit (t = 7.14, p < 10⁻⁵, β = 0.37, 95% CI [0.27, 0.47]), but not for miss trials (t = 0.13, p = 0.9, β = 0.01, 95% CI [−0.16, 0.19]; Fig. 6B). These results indicate that face intensity influenced confidence reports in hit trials only, consistent with limited confidence sensitivity when reporting on the absence of a stimulus^26,60,61.

Fig. 6: Confidence judgments in experiment 2.

A Participants were probed on a 3-point scale for their decision confidence after providing the detection response. Confidence reports were binarized by pooling “moderately sure” and “unsure” responses as “low confidence” and reporting “very sure” responses as “high confidence” (see Supplementary Fig. 6B). B Mean and individual percentage of high confidence trials as a function of stimulus intensity (threshold vs. suprathreshold, in light and dark color shades) for hit trials (left panel; blue) and miss trials (right panel; red; n = 30). C Averaged HGA traces for hit trials as a function of confidence and stimulus intensity across all responsive channels in the VVC (left panel) and IFC (right panel). The black bars indicate significant main effects of confidence (corrected for false discovery rate across time). D AUROC for decoding high vs. low confidence in hit trials, as a function of time around the maximum of the decoded latent variable. All error bars and shaded areas indicate 95% confidence intervals, except in D where shaded areas indicate above chance-level performance at the group level (assessed by permuting labels across trials, uncorrected).

We investigated whether brain activity in experiment 2 reflected confidence. Some responsive channels (131 channels, 13.07%, p < 10⁻⁵, Cohen’s h = 0.29, 95% CI [10.99%, 14.94%]) displayed more HGA in high confidence than low confidence hit trials. Hierarchical mixed-effects regressions with confidence and stimulus intensity as predictors showed that higher confidence was associated with more HGA in three ROIs (VVC: t(4862) = 2.44, p = 0.015, β = 0.023, 95% CI [0.004, 0.041], BF₁₀ = 2.33; SPC: t(2346) = 2.86, p = 0.004, β = 0.039, 95% CI [0.012, 0.066], BF₁₀ = 2.78; IFC: t(8615) = 3.88, p = 1.07 × 10^-4, β = 0.025, 95% CI [0.012, 0.037], BF₁₀ = 10; Fig. 6C) but not in the DLPFC (t(2804) = 0.98, p = 0.33, β = 0.013, 95% CI [-0.013, 0.038], BF₁₀ = 0.44, see Supplementary Figs. 3D and 13A). In miss trials, no statistically significant effect of confidence was found in any of the regions (VVC: t(3081) = 0.72, p = 0.47, β = 0.006, 95% CI [−0.011, 0.023], BF₁₀ = 0.2; SPC: t(1475) = 0.86, p = 0.39, β = 0.015, 95% CI [−0.019, 0.049], BF₁₀ = 0.4; IFC: t(5107) = 1.64, p = 0.1, β = 0.011, 95% CI [−0.002, 0.024], BF₁₀ = 0.41; DLPFC: t(1716) = -0.15, p = 0.88, β = −0.002, 95% CI [−0.024, 0.021], BF₁₀ = 0.2, see Supplementary Figs. 14C). These results indicate that several ROIs, including the VVC, encoded confidence in seeing a face. The effect of confidence appeared to be reflected in a higher maximum of HGA, while stimulus intensity modulated the slope of the HGA increase (Fig. 6C), consistent with a recent proposal that confidence is computed as the difference between the maximal accumulated evidence and the detection threshold^17,26. In this proposal, confidence is not a linear function of the maximal accumulated evidence: if evidence remains below threshold, a higher maximal accumulated evidence should lead to less confidence in not seeing a face.

We next investigated whether the multivariate signal identified in experiment 1 could also explain confidence judgments. To characterize the temporal dynamics with which confidence is built, we assessed how decoders predicted participants’ confidence in having seen a face before, around and after the maximum of the decoded latent variable (Fig. 6D). In the VVC, decoding performance across participants was significant during a window of ~300 ms centered on the maximum, and then went back down to chance-level. These findings support the hypothesis that confidence judgments result from the difference between a detection threshold and the maximal level of accumulated evidence^17,26. By contrast, confidence was better predicted before maximal evidence was reached in the SPC, and after this point in the IFC and DLPFC.

A leaky evidence accumulation process reproduces the neural activity

Our empirical results converge to indicate that the VVC accumulates evidence pertaining to detection and confidence, irrespective of reports. To more formally establish the role of evidence accumulation for visual perception, we assessed if a computational model based on a leaky evidence accumulation process could reproduce both the behavioral and neural data we observed^17,26. Compared to accumulation models that sought to characterize discrimination responses, the model included a leakage term which circumvents the continuous accumulation of noise in a detection task with uncertain timings of stimulus onset^24,62,63 (see Methods for more details). Note that in this model, the accumulation of evidence is not terminated after the perceptual threshold is crossed: the leakage term is the parameter that eventually brings the state of the accumulator below threshold. We simulated 10,000 traces of evidence accumulation and the corresponding detection responses and response times. We could thereby reproduce not only the behavioral results (Supplementary Fig. 15) but also the neural data (Fig. 7B) which resembled several idiosyncratic features of the HGA responses observed in the VVC (Fig. 7A). These include the slightly lower maximum observed for slower hit trials in experiment 1, as well as the effect of intensity in hit trials but not in miss trials in experiment 2. Model simulations therefore support the role of leaky evidence accumulation in stimulus detection (see also Supplementary Fig. 16 for an additional analysis based on model simulations indicating that the VVC’s latent variable dynamics are consistent with the leaky evidence accumulation process). Importantly, note that this computational mechanism is most likely to be implemented by large scale neural dynamics involving distributed brain regions beyond the VVC²⁶.

Fig. 7: Model simulations reflect HGA in the VVC.

A Averaged activity in the VVC in the three experiments, reproduced from Figs. 2B and 3D–F for comparison with the simulation results. Shaded areas represent 95% confidence intervals. B The corresponding simulation traces averaged across repetitions. Traces are plotted as a function of detection and reaction times in experiment 1 (top panel), of detection and intensity in experiment 2 (middle panel) and of intensity in experiment 3 (bottom panel).

Discussion

We established that neural activity in the VVC verified three pre-registered hypotheses: it relates to evidence accumulation in experiment 1, discriminates hits from misses in experiment 2 and is modulated by stimulus intensity during passive viewing in experiment 3. Amongst our four pre-registered ROIs, both the VVC and IFC displayed dynamics consistent with an evidence accumulation process for face detection in experiment 1, as shown by two markers of evidence accumulation. First, reaction times explained the slope of the HGA in responsive channels in these regions. Second, reaction times correlated with the time at which an estimated latent variable, tuned to distinguish detected and undetected faces, crossed a threshold. Hierarchical mixed-effects regressions applied to responsive channels showed that while activity in all ROIs discriminated between hit and miss trials in experiment 2, only the VVC showed a statistically significant influence of stimulus intensity during passive viewing in experiment 3. This result was further substantiated by multivariate analyses showing that the signal underlying detection-sensitivity in the VVC in experiment 1 could generalize to delayed (experiment 2), passive (experiment 3) and non-veridical face perception (experiment 2). Finally, univariate analyses indicated that participants’ confidence in having seen a face was encoded in the VVC, SPC and IFC. Interestingly, our decoding algorithms trained to distinguish between hit and miss trials in experiment 1 could decode confidence judgments at the time of maximal accumulated evidence in hit trials in experiment 2 in the VVC and SPC, but not in the IFC. We note that these results are likely to be valid in the general population, given that sEEG recordings concern mostly healthy cortical tissue⁶⁴.

Widespread evidence accumulation indicates different functional roles of accumulators in the brain

First, our results bring insights regarding the distribution of putative neural accumulators^26,42,65. Previous electrophysiological research in mice^32,35, as well as fMRI^30,31,33,36, MEG³⁷ and intracranial EEG^27,28,66 research in humans uncovered widely distributed accumulator dynamics with different activity profiles in discrimination tasks (Supplementary Fig. 5). This temporal hierarchy has been modeled to rely on local circuit differences and connectivity patterns, in turn impacting global brain dynamics⁶⁷. Our results are consistent with recent evidence in rodents that cortical evidence accumulation is also widespread and diverse in detection tasks²⁵. Exploratory analyses on all brain areas identified six regions reflecting evidence accumulation apart from the IFC and the VVC: the lateral visual cortex, the insula, the dorsolateral motor cortex, the dorsolateral premotor cortex, the ventral somatosensory cortex and the ventrolateral orbitofrontal cortex (Supplementary Fig. 3), consistent with previous research in perceptual decision-making^25,37. Contrarily to putative neural accumulators identified in discrimination tasks, but consistently with what has been previously found in detection tasks^24,26,68 and in our model simulations (Fig. 7B, left panel), we also see an association between onset latencies of the HGA effect and response times (Fig. 2A, B). Importantly, the peak HGA varies more with response times than with onset latencies, as is expected from an evidence accumulation process. Additional model simulations point to variability in the non-decisional time as one possible source for the shift in onset latencies across response times (Supplementary Fig. 17). While the best way to recognize evidence accumulation signals is still under debate^69,70, we found converging evidence for it in the VVC from univariate and multivariate analysis as well as computational modeling.

All regions reflecting evidence accumulation were also found to reflect stimulus detection when the report was delayed in experiment 2. These regions were thus not merely involved in the immediate execution of a motor action, even if the results of experiment 2 do not rule out the possibility that detection signals reflect motor planning for a delayed response^21,58,59. In previous studies, experimental manipulations (e.g., temporal delay, change of effector) made it possible to isolate effector-independent accumulators in the brain, displaying accumulation for abstract decisions^27,71. Going one step further, we tested whether accumulators were involved in perceptual processing when participants were not required to perform any task. We asked participants to pay attention to faces during this experiment even if no task was enforced. Importantly, even if their attention levels were lower than in the previous two experiments, inattention to face stimuli would reduce statistical power for identifying an effect of suprathreshold vs threshold faces on neural activity. Previous works using scalp EEG have shown that centro-parietal markers of evidence accumulation thought to reflect subjective perception⁷² vanish during passive viewing^12,13,71. Crucially, however, in the ventral and lateral visual cortices, we found stimulus-locked activity during passive viewing with similar timings as activity reflecting evidence accumulation during stimulus detection. Additionally, electrical stimulations of bipolar contact pairs in the VVC of the 4 participants for whom we could successfully decode detection reports did not lead to any motor action or motor-related sensation, further indicating that this region did not play a role in motor planning. Scalp EEG studies may fail to capture the activity we observed in the VVC due to its anatomical localization below the temporal lobe (see below).

Evidence accumulation in the absence of report was most evident in the mid-fusiform gyrus

Because of the uncontrolled and sparse sampling of sEEG, not finding evidence for neural signatures in other ROIs than the VVC should not be interpreted as conclusive evidence for their absence, and direct comparisons between regions should be avoided⁶⁴. An additional concern in experiment 3 is that participants might have been paying limited attention to stimuli, thereby reducing statistical power to detect significant effects. This is particularly problematic in the SPC, as we could not collect data in experiment 3 for two participants who showed decoding of delayed detection in experiment 2 in this region (Supplementary Table 3). Previous studies showed that neurons in the SPC reflected evidence accumulation in both detection and discrimination tasks^26,34 and encoded the intensity of vibrotactile stimuli in a no-report²⁶. In the IFC, Bayesian analyses on the effect of stimulus intensity on HGA in experiment 3 remained inconclusive. Although markers of evidence accumulation were found in 7 participants, none of the decoders could generalize to the passive viewing condition in experiment 3 despite some single channels showing a strong effect of stimulus intensity (see Fig. 3E). Note that these channels appeared to be specifically located in the anterior insular cortex at the junction with the frontal operculum (see Fig. 4B and Supplementary Fig. 10), an area that was responsive to stimulus perception in previous no-report studies^56,73. There is a possibility that decoding did not transfer from experiment 1 to experiment 3 in this region due to the sparse sampling of sEEG. Alternatively, our decoders may have failed to generalize from experiment 1 to experiment 3 because of the algorithm they were based on. This algorithm assumes that stimulus detection is determined by the maximum amount of accumulated evidence, while insular and frontal evidence accumulation processes might reflect processes linked to reports (but see ref. ⁷⁴). Decoding transfer was observed in the VVC, and appeared to be driven by channels located around the mid-fusiform gyrus (Fig. 4A and Supplementary Fig. 10). This cortical location, which includes the fusiform face area, has been found to be responsive to faces using several imaging methods^75,76,77,78, to be involved in the recall of faces⁷⁷, and to lead to distorted perception of faces when stimulated^54,64,75. This specificity can explain why stimulus detection in experiment 1 was decoded in only 4 participants out of 17 in the VVC, all of whom showed high decoding accuracy (AUROC ~ 80%) and reflected evidence accumulation. Notably, all 4 participants had channels located in the mid-fusiform gyrus, and the average distance of channels from the fusiform face area (average coordinates from ref. ⁷⁹) was smaller for the participants who showed successful decoding (mean distance = 22 mm) than for those who did not (mean distance = 33 mm; t = 5.3, p = 2.63 × 10⁻⁷; unpaired t-test). Because of clinical constraints, we only had data for two out of these four participants (G5 and G18) in experiment 3. Participant G5, but not G18, showed successful decoding of stimulus intensity in experiment 3 (note again that decoding performance for this participant was high, AUROC = 0.69). We see at least two possible explanations for the differences in decoding success across these participants. On the one hand, evidence accumulation for stimulus detection might be performed by some parts of the responsive visual cortex and not others. On the other hand, some participants might have had lower evidence accumulation rates or higher perceptual thresholds during passive viewing in experiment 3, and therefore failed to see most faces. Anecdotally, in a short passive viewing session at the end of the experiment where inverted faces were introduced to assess whether participants still experienced the faces and during which participants provided no behavioral response, G5 noticed the presence of inverted faces while G18 did not notice them. Note that G22, for whom stimulus intensity in experiment 3 could be decoded in the lateral visual cortex, also noticed the inverted faces.

Implications for perceptual consciousness

No-report tasks like the one we used in experiment 3 to disentangle between neural activity reflecting perceptual processes and task-related processes have been used to draw conclusions about the neural correlates of perceptual consciousness^{5,8,12,13,14,73,80,81,82,83}. Arguably, the HGA signals observed in the VVC in experiment 3 are indicative that this region plays a role in subjective perception, consistent with fMRI⁷³, intracranial^84,85 and primate electrophysiology⁸¹ studies showing activity in occipitotemporal regions in visual no-report paradigms. However, this conclusion faces some limitations. First, we cannot fully discard the possibility that the results reflect post-perceptual cognitive processing^86,87. A stimulus can trigger covert decisions or mind-wandering even when the observer is not instructed to perform a task on it. These processes would only occur when participants see the stimulus, and not when they fail to see it, thereby contaminating the results. Nonetheless, no-report tasks in scalp EEG experiments have proven crucial in determining that brain signals which were considered markers of perceptual consciousness actually reflected task-related processes^12,83. Similarly, we found many regions that reflected detection when a report was required from the participant in experiment 2, but not during passive viewing, indicative of task-related activity. We also note that findings from experiment 3 cannot be interpreted in isolation. Indeed, they result from comparing VVC activity in two conditions where stimuli were presented at different intensities, which could therefore reflect sensory encoding independent of stimulus detection. Importantly, though, the same channels in the VVC also reflected stimulus detection in experiments 1 and 2, and a classifier trained to distinguish between detected and undetected stimuli in experiment 1 could decode stimulus intensity in experiment 3. More research is required to determine if VVC signals reflect perceptual consciousness. It will be important to invert the order of report and no-report conditions to rule out that participants engaged in post-perceptual cognitive processing after having performed the task extensively⁸⁶. It will also be interesting to cross-validate our results with oculometric measurements, which provide a way to classify whether a stimulus is perceived in no-report conditions without relying on different stimulus intensities^5,47,81,88.

Slow accumulation for threshold stimuli produce late markers of stimulus detection in the VVC

In our detection tasks, the onset of the HGA detection effect in the VVC was consistent across experiments (0.33 s after stimulus onset in experiment 1 and 0.34 s in experiment 2) and reflected late reaction times (0.74 ± 0.11 s, with only 2.16% of trials under 0.4 s). Late timings are consistent with what was observed in single neurons in the parietal cortex in a vibrotactile detection task using threshold stimuli²⁶. Despite these late timings, our modeling results suggest that evidence accumulation could be starting as early as 150 ms after stimulus onset. Our findings thus raise the possibility that the neural activity underlying detection occurs at different latencies after the stimulus onset according to both the drift (deterministic) and the diffusion (stochastic) components of the evidence accumulation process. In experiment 1, we observed that the HGA increase in the VVC was markedly delayed for slower responses times (i.e., varied according to the diffusion component). In experiment 2, we did not have access to response times but we observed an interaction effect between detection responses and stimulus intensity, suggesting that the increase in HGA was delayed following threshold compared to suprathreshold stimuli. Accordingly, while non-degraded face stimuli should have strong drift rates and lead to early increases of neural activity in the VVC^{48,54,77,84,85,89,90,91,92,93,94}, degraded faces at the perceptual threshold in our study are associated with low drift rates that delay neural activity correlating with stimulus detection. Our results, therefore, call for a nuanced interpretation of latencies, particularly as a diagnostic tool for assessing theories of consciousness⁹¹.

A leaky evidence accumulation process can account for the pattern of results observed in the VVC

It has been proposed that contents reach consciousness when an accumulated signal crosses a threshold^17,39,41,95, and a growing body of evidence has linked evidence accumulation to perceptual consciousness^26,40,72,96. Our study adds to this body of work: The neural code that was found to reflect evidence accumulation in the VVC also reflected stimulus detection during passive viewing, and the occurrence of false alarms was found to correlate with the number of times that the decoded latent variable crossed a detection threshold in the VVC (Fig. 5). These results fit into a recent proposal that several aspects of perceptual consciousness are determined by a leaky evidence accumulation process¹⁷. To formally gauge whether it accounts for stimulus detection in our study, we assessed if simulations based on a computational model instantiating this process could reproduce the observed pattern of data. Despite being trained on behavioral data only, this model reproduced several idiosyncratic features of the HGA in the VVC (Fig. 7). The results therefore provide support for the proposal that stimulus detection is determined by a leaky evidence accumulation instantiated in high-level sensory regions. At this stage, this process appears compatible with several of the main candidate theories of consciousness¹⁷, although it will be helpful to refine their experimental predictions to determine how exactly evidence accumulation fits into them⁹⁷.

Confidence computations in the VVC are related to the maximum of accumulated evidence

Some authors have argued that perceptual consciousness consists not only of perceptual properties, but also of distinct epistemic feelings of confidence associated with perceptual monitoring^15,17. Our results show that the same neural code reflects stimulus detection and perceptual confidence. Perceptual confidence was found to relate to the maximal level of evidence that is reached after the threshold is crossed when participants see a face. Consistent with the view that confidence derives from maximal evidence^17,26, decoding of confidence in seeing a face in the VVC was best around the time of the maximum of the latent variable (Fig. 4E), an indication that this information was important in confidence computations. Our data indicated that the SPC and IFC also reflected perceptual confidence, but hinted at different computations across regions. Contrary to what we observed in the VVC, we found that confidence was best predicted early in the trial in the SPC. In the IFC, a region that has been argued to accumulate evidence for confidence^28,98, confidence could not be decoded at the time of the maximum of the latent variable. These results indicate that confidence is represented differently in those regions compared to what is observed in the VVC and what is predicted by the leaky evidence accumulation process. An interesting avenue for future research will be to disentangle the roles of those different regions and their interactions in how confidence computations unfold.

To conclude, we showed that a neural code in the ventral visual cortex reflected evidence accumulation for face detection, also when reports were delayed or absent, confirmed by a computational model based on a leaky accumulation process that could reproduce traces of high gamma activity in all three experiments. Additionally, it could decode confidence judgments in seeing a face. Altogether, our results indicate that evidence accumulation occurs beyond processes relating to preparing and executing a report and could explain the processes associated with perceiving a stimulus. This proposal also implies that stimulus perception might occur at varying latencies after stimulus onset, depending both on its physical intensity and the stochastic nature of the associated sensory evidence.

Methods

Pre-registration and deviations

The pre-registration for this study was uploaded on February 28, 2022 on the Open Science Framework (https://osf.io/8qe3a/).

Deviations from the pre-registered plan and the rationale for these deviations are the following:

• The nomenclature of two ROIs was changed from fusiform gyrus and inferior frontal gyrus to ventral visual cortex and inferior frontal cortex to better reflect the specified regions (VCcm, ITCm, and ITCr, and PMrv and PFrvl in MastAtlas, respectively).

• Data collection slightly exceeded the defined stopping rule criterion, due to delays in obtaining the final electrode positions for some patients.

• The proposed method for identifying sensitive channels and ROIs - time point by time point analyses corrected with the false discovery rate for each contrast - was suboptimal because it was too stringent. Instead, we identified responsive electrodes by performing nonparametric clustering analyses for the hit vs. miss contrast in experiment 1 and ran subsequent analyses on identified positive clusters only. This approach presents the benefit of maintaining the same false discovery rate, while increasing the statistical power of the analyses. The results for the original approach are reported in the Supplementary Note 1.

• The pre-registered confidence models in experiment 2 were flawed in that they did not mention stimulus intensity as predictor. It was added to ensure that confidence effects could not be explained by low confidence trials having a higher proportion of threshold stimuli. Because of model convergence issues for the full model (confidence*intensity*detection), separate models (confidence*intensity) were run for hit and miss trials.

• Based on a suggestion made in the reviewing process, we conducted Bayesian mixed-effects regressions and computed Bayes factors to quantify evidence for the null hypothesis.

Participants and data acquisition

29 participants (10 females (self-report), aged 36 ± 9 years; Supplementary Table 1) with drug-resistant focal epilepsy were included across two separate sites, the Grenoble Alpes University Hospital, France (n = 22) and St. Anne’s University Hospital Brno, Czech Republic (n = 7). Sex and gender were not considered in the study design or analyses, as the findings do not apply to only one sex or gender. No statistical method was used to predetermine sample size. All included participants participated in experiments 1 and 2, and 27 (9 females, aged 35 ± 8 years) out of the 29 participants participated in experiment 3. Participants were on the following antiseizure medications during their hospital stay: Lacosamide (n = 7), Perampanel (n = 9), Carbamazepine (n = 10), Lamotrigine (n = 10), Zonisamide (n = 5), Zebinix (n = 1), Oxcarbazepine (n = 1), Levetiracetam (n = 4), Clonazepam (n = 2), Brivaracetam (n = 6), Eslicarbazepine (n = 4), Topiramate (n = 2), Cenobamate (n = 1), Pregabaline (n = 1) They were implanted with 10 to 18 stereotactic electrodes in the context of presurgical evaluation, and electrode placement was conditioned on clinical criteria. Electrode implantation and participation in research protocols complied with all relevant ethical regulations and received ethical approval from CPP du Sud-Ouest et Outre-Mer 4 CPP18-001b/2017-A03248-45 (in Grenoble) and Ethical committee at St. Anne’s University Hospital 46 V/2019 (in Brno). All participants provided their informed consent and no financial compensation was provided for participation. They were implanted with semirigid, multilead electrodes of 0.8 mm diameter, containing up to 18 contact leads 2-mm-wide and 1.5-mm-apart (DIXI Medical® USA). Neural recording in Grenoble was conducted at 512 Hz with the Micromed (Trevisio, Italy) audio-visio-EEG recording system. Neural recording in Brno was conducted at 5000 Hz with the M&I (BrainScope, Prague, Czech Republic) 192-channel research EEG acquisition system, and downsampled to 512 Hz before preprocessing. The anatomical locations of electrode contacts in both sites were identified on BrainVisa⁹⁹ using a post-implantation CT scan co-registered with a pre-implantation T1 MRI scan.

Four participants were excluded from all analyses because they did not respond directly after stimulus presentation in experiment 1 and/or had a higher rate of false alarms than hits in experiment 2. All participants performed the three experiments, except for two (G2 and G8) for whom we could not acquire electrophysiological recordings in experiment 3 due to clinical constraints. Finally, one participant (G5) came back to the hospital with a new implantation scheme and was tested again on this occasion. We report the results for the two sessions from this participant as two independent datasets.

Stimuli

Stimuli were presented on a Dell Precision 7550 laptop (15 inches), with 1920 × 1080 pixels resolution and 60 Hz refresh rate in Grenoble, and on a 27 inches LED display with 1920 × 1080 pixels resolution and 120 Hz refresh rate in Brno. We checked that the behavioral results were similar in the two data collection centers despite the different equipment used by adding the recording center as a variable of no-interest in a control analysis. The fixation cross was presented at the center of the screen, and each of its four arms was 10 pixels long and 2 pixels wide. Mask frames were also centered on the screen and measured 768 × 768 pixels. Frames were presented for a duration of 200 ms, and the scrambling procedure described below was done afresh for each frame. For each trial, the masks were generated using one of four greyscale pictures of neutral-valence human faces (2 males, 2 females) that were cropped such that they contained only facial features and blurred around the eyes and mouth. Face stimuli do not represent actual people: they are AI-generated and freely available for scientific purposes on this website: https://generated.photos. Scrambled images were obtained by randomly permuting the phase angle of the discrete Fourier transform of the images, on which an inverse discrete Fourier transform was applied. The phase scrambled stimulus generated for each frame was combined with that of the original face image at an intensity equal to I, using I*face + (1-I)*mask.

Face stimuli in target-present trials in experiment 1 were all presented at detection threshold (threshold trials) for three frames (600 ms). In experiments 2 and 3 faces could be presented at threshold or above threshold (suprathreshold trials). Intensity I in suprathreshold trials was 1.25 and 1.5 times the threshold value in experiment 2 and 3, respectively. Detection threshold was determined using a 1-up/1-down staircase procedure titrating 50% hit rate, with an initial I value of 0.15. Stimulus intensity was increased/decreased by a percentage of its current value (5%) after each hit/miss response in target-present trial. In order to account for possible threshold drifts during the experiment, the I value was also continuously adapted in experiment 2 after each threshold trial by an increase after “seen” and a decrease after “not seen” responses. The step in experiment 2 was 5% for participants B1 to B7 and G1 to G8, and was 2% for participants G9 to G21 to improve staircase convergence.

In target-absent trials, the I value was set at 0 for all mask frames. In experiment 2, there were some trials (one per block) for which a second face stimulus was presented. These trials were introduced in order to incentivize participants to use the “more than one face” response option in order to assess non-veridical perception of a second face when only one was presented. Each face stimulus was presented on three consecutive frames, and the onset of the second stimulus occurred 600 ms after the offset of the first stimulus. In those trials, the two faces were always presented at suprathreshold value (i.e., 1.25 times the threshold value), and also always had the same onset relative to the beginning of the trial (i.e., the presentation of the first mask frame), at 200 and 1400 ms.

Procedure

Participants were seated in a hospital bed, with the stimulation laptop in front of them on a bedside table. The experiments were run with Psychtoolbox-3¹⁰⁰ on Matlab 2019b. The stimulus sequence was the same in all three experiments. First, participants were presented with the fixation cross on a gray screen background for 600 ms. Then, 13 consecutive mask frames were presented at 5 Hz for 2600 ms. A face stimulus was embedded for a duration of 600 ms starting at a random frame from 600 to 1800 ms after the onset of the sequence. This sequence was followed by a random delay of 500 to 1000 ms during which no stimulus was presented. In experiment 1, participants were asked to press a key as soon as they saw a face. In experiment 2, participants first provided a detection response following a prompt which remained on screen until a response was made. The second answer screen probing for a confidence response was presented 100 ms after the first response and until the participant responded. The next trial started 100 ms after the confidence report. In experiment 3, participants passively observed the stimuli. Each experiment is explained in further details below.

Before taking part in the three experiments, participants were presented with a five-trials long demonstration with the same structure as experiment 2. In the first trial, the embedded face stimulus was presented at maximum intensity (i.e., no phase scrambling at all). In subsequent trials, the intensity was reduced by steps of 20% so that they understood the difficulty of the task. Finally, in the last trial, the target was absent. Participants were instructed to wait for the answer screens to provide detection and confidence reports. If the instructions were still unclear to the participant after this short sequence, the demonstration was repeated once again before starting experiment 1.

Experiment 1: Participants were asked to provide an immediate response as soon as they saw a face and not to respond if they did not see a face. They responded by pressing the button 2 on a numerical keyboard with the index of their right hand. No feedback was given on their performance and they directly proceeded to the next trial. Participants performed two blocks of 52 trials in Grenoble and 40 trials in Brno, including ~20% target-absent trials (10 and 8 trials in Grenoble and Brno, respectively).

Experiment 2: Participants were instructed to respond at the end of the trial, when a question displayed in the middle of the first answer screen probed them for a detection report (“How many faces did you see?”). They responded by pressing one of three buttons on a numerical keyboard corresponding to “none”, “one”, or “more than one”. They were informed that in some trials two faces may be presented. These trials occurred only once per block. Only “one face” responses were treated as hits or false alarms when a single face was presented during the trial. A second answer screen then asked participants to report their confidence on a three-point scale conditioned to the detection report (“Confidence of having seen no face” or “Confidence of having seen one face or more”). They responded by pressing another set of three buttons corresponding to “unsure”, “moderately sure”, or “very sure”. Participants were encouraged to make full use of the confidence scale, both when they had reported seeing a face and when reporting seeing no face. When they reported seeing “more than one face” to the detection question, they were asked to report their confidence based on the clearest face stimulus they had perceived. No time constraints were applied on responding to either the first or the second question (i.e., the prompts remained on screen until participants provided a response). In Grenoble, participants performed three blocks. In Brno, they performed two blocks. Each block consisted of 59 trials, including 10 target-absent trials and one trial with two faces. Target-present trials were constituted of an equal number of threshold and suprathreshold trials.

Experiment 3: The stimulus sequence was the same as in the previous two experiments, yet here, participants were not asked to report anything. Instead, they were simply instructed to keep fixation and remain attentive to the stimulation. There were no target-absent trials. Akin to experiment 2, there was an equal number of threshold and suprathreshold trials. Given that no answers were provided, stimulus intensity was not titrated. The number of blocks was identical to Experiment 2, with 48 trials in each block.

To test if participants were attentive to the visual stream, they were presented with a short no-report control block at the end of experiment 3. The block consisted of 12 trials, with 8 upright and 4 inverted face stimuli (equal number of threshold and suprathreshold trials). We assumed that if they saw them, the inverted faces would have been noticed by the participants because all of the faces they had been presented with so far were upright. At the end of the block, the experimenter asked the participant if they noticed anything odd about the last block. If they responded that it was the case, they were further asked what that was. A record was kept of whether they reported seeing inverted faces or not.

Behavioral analysis

Trials with early or late responses in experiment 1 were excluded from all analyses. A response was considered early if it occurred earlier than 400 ms prior to stimulus onset and as late if it occurred later than 1500 ms after stimulus onset. On average, these responses accounted for 2.16 ± 2.32% and 0.86 ± 2.03% of trials, respectively. In experiment 2, trials with response times over 10 s were excluded (0.54% of trials), as it was likely that the participant was not performing the task anymore if they took so long to respond. We also excluded trials where more than one face was presented and trials where participants reported seeing more than one face (6.87% of remaining trials where one face was presented).

Detection and confidence responses in experiment 2 were fitted with effects-coded generalized linear mixed effects regressions with an underlying binomial distribution and a logit link function (fitglme function in Matlab). For detection responses, the relative intensity of the stimulus (threshold vs. suprathreshold) was specified as fixed effect factors and participants as random effects. For decision confidence responses, the fixed effects were relative intensity, detection response and their interaction. Separate models were then fit for hit and miss trials. We found that most participants reported a majority of “very sure” responses (Supplementary Fig. 6). To avoid design cells containing very few trials, we split the scale in two, reporting “very sure” responses as “high confidence” and pooling “moderately sure” and “unsure” responses as “low confidence”. Log likelihoods were compared to determine whether to add random coefficients to each model.

Electrophysiological data processing and analysis

Preprocessing: The location of contacts was determined using a pre-implantation MRI and a post-implantation CT scan on the Intranat software¹⁰¹. MarsAtlas parcellations were performed in participants’ native brain space¹⁰². Coregistration to MNI space was performed in Intranat using SPM12.

Electrophysiological signals were visualized and analyzed under Matlab 2020b using custom-made scripts and the FieldTrip toolbox¹⁰³. Raw signals were visually inspected to exclude channels with artifactual or epileptic activity (24.03% of all contacts). A notch filter was then applied at 50 Hz, 100 Hz and 150 Hz to eliminate power line artifacts. Bipolar derivations were obtained by re-referencing neighboring contacts to each other. When one isolated contact was excluded, we computed a derivation for its two neighboring contacts. Of 5262 recording contacts, 3572 channels remained after inspection and bipolar derivation. Channels that were not located in the MarsAtlas (n = 271) were excluded from all analyses.

High-gamma activity (HGA) was computed based on the electrophysiological signal between 70 and 150 Hz. Non-causal filters were applied on each trial to define 20 Hz overlapping frequency sub-bands (70 to 90 Hz, 80 to 100 Hz, etc.), whose envelope was extracted with a Hilbert transformation. The extracted signal was then divided by the mean activity starting one second before the trial and ending at trial onset, and averaged across sub-bands. The resulting signal was smoothed using a quadratic Savitzky-Golay filter¹⁰⁴ of polynomial order 2 with a 200 ms window to preserve the peak amplitudes. Note that dividing with baseline activity means that the signal is centered around 1. Finally, the data were resampled at 64 Hz. Individual trials with remaining artifacts were removed by visual inspection (7.21% of trials).

Anatomical definitions: We limited our principal analyses to four pre-registered ROIs: the VVC, the SPC, the IFC and the dlPFC. ROIs were defined based on combinations of regions from the MarsAtlas cortical parcellation model (Auzias et al., 2016). The VVC (234 channels) corresponded to the caudal medial visual cortex, medial inferior temporal cortex and rostral inferior temporal cortex in MarsAtlas. The SPC (72 channels) corresponded to the superior parietal cortex and medial superior parietal cortex, the IFC (267 channels) to the rostral ventral premotor cortex and rostral ventrolateral prefrontal cortex, and the dlPFC (103 channels) to the rostral dorsolateral inferior prefrontal cortex and rostral dorsolateral superior prefrontal cortex. Responsive channels in ROIs were visualized in native space by a trained neurologist (A. R.). To make sure of the anatomical origin of the observed effects, we performed two sets of control analyzes. First, because some channels classified as belonging to the IFC were actually located in the anterior insula, we looked at a control region encompassing the anterior insular cortex and the functionally homologous internal operculum. Second, we looked at the impact of excluding channels that were located in the white matter. We also performed exploratory analyses on all regions in the MarsAtlas that included more than 30 channels and at least 10 responsive channels in the hit vs. miss contrast in experiment 1 (21 out of 31 possible regions). Plots of brain slices were created using SPM12¹⁰⁵.

Univariate analyses: We performed nonparametric clustering analyses on individual channels for hit vs. miss trials or threshold vs. suprathreshold trials⁵⁷. Responsive channels (α = 0.05) were identified as those showing a positive cluster (i.e., HGA in hit trials > HGA in miss trials) in experiment 1. In the evidence accumulation analysis, HGA slopes were calculated by fitting a linear regression model starting 200 ms after stimulus onset and stopping 100 ms before the response. HGA slopes were then fitted to reaction times using linear mixed-effects regressions.

For analyses based on ROIs, in experiments 2 and 3, the HGA of each responsive channel was averaged across time points for each condition within the positive cluster identified for that channel. This averaged HGA was then fitted with generalized linear models with an underlying gamma distribution and a log link function. Time-resolved results were obtained by fitting a model on each time point in a one-second window following stimulus presentation and correcting across time points using the false discovery rate procedure¹⁰⁶. In experiment 1, HGA slopes were fitted with hierarchical mixed-effects regressions, considering that slopes followed a normal distribution under the null hypothesis. To account for the full hierarchical structure of the data, all analyses on ROIs involved hierarchical models with random effects for channels and participants. As an example, the model using detection and stimulus intensity as predictors in experiment 2 had the following structure:

$${{{\rm{HGA}}}} \sim {{{\rm{detection}}}} \,\ast \,{{{\rm{intensity}}}}+({{{\rm{detection}}}}+{{{\rm{intensity}}}}\,|\,{{{\rm{participant}}}}) \\+({{{\rm{detection}}}}+{{{\rm{intensity}}}}\,|\,{{{\rm{channel}}}}:{{{\rm{participant}}}})$$

To quantify evidence in favor of null hypotheses in ROIs, we estimated Bayesian mixed-effects regression with the brms package in R¹⁰⁷. Model structure was the same as for the frequentist models. Priors were determined empirically for each factor separately in a two-step procedure: (1) Fitting of the data from another region (the VCl) with uniform priors, and (2) fitting the data of the ROIs with a prior informed by the posterior obtained in step 1. The informed priors were centered on 0 and had a standard deviation equal to 5 times the posterior distribution observed in the VCl. Bayes factors were estimated based on the Savage-Dickey ratio. The inference criterion for the null was BF₁₀ <⅓. All Bayesian models were fitted using 4 Markov Chain Monte-Carlo chains with 5000 iterations each, including warmup 2000 iterations. Bayesian sampling was assessed using R-hat, which were below 1.01, and Effective Sample Sizes (ESS), which were greater than 1000¹⁰⁸. The resulting prior and posterior distributions are shown in Supplementary Fig. 4.

To determine whether the number of channels showing an effect in one condition exceeded what would be expected by chance, we used binomial tests against chance-level (α = 0.05). Confidence intervals were calculated using the Wald method.

Multivariate analyses: Custom decoders were trained to discriminate between hit and miss trials in experiment 1 separately for each participant, on ROIs with five channels or more. They were fed trial-by-trial HGA from a window starting at stimulus onset and finishing 1 s later. The Nelder–Mead method was used to obtain weights that maximized the distance of the maximum values of a time-resolved trial-by-trial latent variable across classes. Specifically, we obtained a distance term for each trial by calculating the logarithm of the normalized distance between the maximum and a threshold. To counteract the effect of class imbalances, the distance term of a given trial was divided by the number of trials in that trial’s class. The error term that the fit function attempted to maximize consisted of the difference between distance terms in class 1 (e.g., hit) and class 2 (e.g., miss). A penalty was applied on the overall error term to promote sparsity (L1 regularization). A Bayesian optimization search was performed to find the best value of that hyperparameter, using a 5-fold cross-validation procedure.

Decoders performing within-sample predictions were trained using a leave-one-out procedure. Other decoders were trained on all trials in experiment 1 to perform out-of-sample predictions on detection response, confidence judgements and stimulus intensity in experiments 2 and 3. In the case of confidence judgment and stimulus intensity, the labels were modified such that a successful prediction corresponded to classifying a high confidence/suprathreshold trial as a hit or a low confidence/threshold trial as a miss. Decoders were only tested when there were at least 5 trials per experimental cell (e.g., low and high confidence). Decoders were tested in the same stimulus-locked window than the one they were trained on (0 to 1000 ms after face stimulus onset), except in analyses including trials without such events to lock the window on, i.e., false alarms or reports of seeing more than one face, in which case the decoders were tested on a window starting at the earliest time that faces could appear and finishing right before the earliest possible onset answer screen (600 ms to 2800 ms following the start of the trial). In confidence analyses, we also tested decoders’ performance in a window of 400 ms centered on the time of the maximum of the latent variable. To assess decoding performance, we reported the area under the receiving operator characteristics (AUROC). We permuted labels 1000 times to assess whether the obtained values for each participant were significant. To correct for multiple comparisons within each region of interest, we performed the false discovery rate procedure across participants¹⁰⁶. For group analyses, we tested observed average AUROC values against a null distribution obtained by averaging AUROC values obtained from permutations. To correct for multiple comparisons, we again applied the false discovery rate, yet this time across regions¹⁰⁶.

We estimated a latent variable for each trial by applying the set of weights from the decoder at each point in time. The threshold obtained in the training procedure did not provide the best categorization of classes, as it was focused on maximizing distances instead. We thus estimated individualized detection thresholds. per participant by taking the threshold value of the receiver operating characteristic curve that maximized the rate of true positives (hit trials correctly classified as such) to false positives (miss trials incorrectly classified as hit). In evidence accumulation analyses, hit trials for which the estimated latent variable crossed the threshold were selected for a Spearman’s rank-order correlation between the timing of this threshold crossing and reaction times. In analyses on false alarms and trials in which participants reported seeing more than one face (Fig. 5), we averaged by condition the number of times that the latent variable crossed the detection threshold computed from hit and miss trials in experiment 2. Statistical significance was also calculated using permutation tests.

Computational model

The model instantiated an evidence accumulation process with a leakage factor^26,63 and included five free parameters: a drift rate with mean Γ and standard deviation s, a flat decision boundary θ, a leakage factor λ, and a non-decision time ndt. A white Gaussian noise process W(t) with zero mean and unit variance was added to the process to account for stochasticity (Eq. 1). Since neuronal responses such as high-gamma activity are positive, the evidence accumulation EA(t) process was bound to zero for biological plausibility.

$${{{\bf{EA}}}}(t+1)=\max ([1{{{-}}}\lambda \,\ast \,{{{\rm{dt}}}}] \,\ast \,{{{\bf{EA}}}}(t)+{{{\bf{d}}}}(t)+\sigma {{{\bf{W}}}}(t),0),\sigma=0.1$$

(1)

The accumulation process started at the beginning of each trial with a zero drift rate. On stimulus onset S, following a non-decision time, d(t) increased to a level (Γ + sZ)log(I + 1), for 600 ms (stimulus duration) before returning to zero at stimulus offset (Eq. 2). I represents the stimulus intensity and Z a zero-mean and unit-variance Gaussian noise varying on a trial-by-trial basis.

$$\begin{array}{cc}0,& {{{\rm{if}}}}\,t < (S+{ndt}) \hfill \\ {{{\boldsymbol{d}}}}(t)=(\varGamma+s{{{\rm{Z}}}}) \, {{{\bf{log }}}}(I+1),& {{{\rm{if}}}}(S+{ndt}) < t < (S+{ndt}+0.6)\\ 0,& {{{\rm{if}}}}\,t > (S+{ndt}+0.6) \hfill \end{array}$$

(2)

Stimuli were considered as detected if the accumulated evidence reached the decision boundary θ. Parameters were adjusted based on pilot data to reproduce behavioral and neural data (Γ = 0.012; s = 0.4*Γ; θ = 3.45; λ = 4; ndt = 0.3).

We simulated 10000 trials including 4000 threshold trials, 4000 suprathreshold trials, and 2000 catch trials (no stimuli). As in the actual experiment, suprathreshold intensity was set to 1.25 times the threshold intensity for experiment 2 and 1.5 times that intensity for experiment 3. The non-decision time was also divided into two components: a pre-decision time of 150 ms and a post-non-decision time of 150 ms. The set of parameters was used to generate traces of accumulating evidence (time-locked to stimulus onset) and simulate detection responses. A fixed random time shift (normal distribution with a mean of 0 and a standard deviation of 50 ms) was applied to each trial’s evidence accumulation trace to simulate non-decision time variability, similar to previous works⁴⁶. Similar to behavioral analysis, simulated response times outside the range of 0.4 s to 1.5 s following stimulus onset (7.6% and 5.4%, respectively) were excluded.

For visualization, we separated simulated hit trial traces (for threshold condition) in three terciles based on reaction times to show the relationship between response times and accumulating evidence in experiment 1. We plotted the average of accumulating evidence in each tercile from 0.25 s before stimulus onset until the median reaction time (Fig. 7B left panel).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.