Introduction

Eye movements and pupil responses shed light on perceptual and cognitive processes by providing continuous indices of information processing and insight into temporal dynamics of perception. Microsaccades, small incessant eye movements occuring ~ 1–2 times per second, have a canonical temporal profile regarding stimulus presentation–rates decrease with respect to stimulus onset and rebound eventually returning to baseline1,2,3–indicating a transition from sensory accumulation to response preparation4.

Lower microsaccade rates and enlarged pupil size have been regarded as markers of difficulty5,6,7,8,9. Here, we used a basic motion discrimination task to investigate whether and how task difficulty impacts the temporal dynamics of the rebound from microsaccade inhibition and pupil size. We hypothesized that difficulty prolongs microsaccade inhibition, resulting in a delayed rebound, as well as in an increase in amplitude and delay in pupil dilatory responses.

Microsaccades serve several roles. They redirect gaze position at a fine scale to improve task performance, correct for ocular drift during sustained fixation, and prevent image fading. In certain contexts, microsaccades are perceptually costly (reviews10,11,12). Microsaccade rate has also been implicated as a marker of cognitive load, as rates are more suppressed when reporting letter shape than color (high and low load demand, respectively13); detecting changes for ambiguous than unambiguous motion14; for tasks requiring greater working memory load15,16; for difficult than easy arithmetic7,8; and when performing visual search tasks with greater difficulty17.

Microsaccade rate modulates over time as a function of stimulus parameters and task-related cognitive processes. Microsaccade inhibition–a canonical temporal characteristic of oculomotor behavior of decreased rates–occurs with respect to an “event” and subsequently rebounds back to baseline rates. For example, after a sudden onset transient, both saccades and microsaccades are inhibited and then rebound eventually returning to baseline rates, consistent with ample evidence that these eye movements lie on a continuum11,18,19,20. Inhibition of (micro)saccades were initially investigated with respect to sudden display changes that result in a quick (~ 50–150 ms) reduction following a transient1. This “reflexive inhibition” is considered an automatic subcortical response driven by low-level sensory processes21,22; the latency and magnitude of this suppression depend on signal properties such as contrast, size, spatial frequency, and sensory modality for microsaccades3,21,23 and saccades24,25. This reflexive inhibition occurs after transient visual or auditory signals26,27 but only when perceived28.

Microsaccade inhibition can also be non-reflexive, modulated by cognitive processes. First, inhibition is more pronounced during task engagement. Temporal expectation of visual29,30,31, auditory32 or tactile33 events reduces microsaccade rate prior to a stimulus, and temporal attention34,35,36 exaggerates and accelerates this inhibition. The magnitude and duration of inhibition also depends on the frequency/saliency of a task-relevant stimulus37,38. Second, rates depend on whether the observer is sustaining fixation or free viewing. Sustained fixation results in higher microsaccade rates39. During free viewing, microsaccade rates increase with scene and task complexity, as exploring a scene18 or precisely relocating gaze39 can be task beneficial, but rates nonetheless decrease during the decision-making stage17. Third, rates may depend on whether the stimulus or task is time constrained. As inhibition corresponds to sensory evidence accumulation, time restrictions (either by design or self-imposed) are likely to affect “processing time”. If an observer has longer access to a stimulus or has slower response tactics, the inhibition differences may be less pronounced because the visual system is less challenged to accumulate evidence and respond fast.

Rebound from suppression is the enhancement of microsaccade generation after suppression. Rebounds typically occur ~ 350 ms after a transient onset, as a result of “reflexive inhibition”. However, cognitive factors can influence the latency and extent of this enhancement as rebounds are not necessarily reflexive and may involve indirect cortical pathways21,40,41. The dynamic shifts of microsaccade rebound have not yet been investigated with respect to task difficulty. We hypothesized that task difficulty in motion discrimination lengthens sensory evidence accumulation periods, which would lead to later discrimination response times, and delayed rebounds from inhibition.

Pupil size (9,42; review43) and microsaccade rates 44 index arousal, which increases with task difficulty. Recent evidence linking microsaccade and pupil responses suggest a potential joint delay in both oculomotor metrics. Pupil size and microsaccades are larger during difficult than easy arithmetic45; and pupil size increases and microsaccade rates decrease when detecting speech under higher than lower noise levels46. Their linked temporal dynamics support a common neural source47,48,49. Given these findings, we additionally hypothesize that task difficulty will increase and delay pupil responses.

We tracked eye position and pupil size as observers discriminated the motion direction of a drifting grating briefly presented in the perifovea while fixating on a central marker. This task is well-suited to assess the temporal dynamics of microsaccades as a function of task difficulty, as small eye movements can introduce motion signals that interfere with the perception of a moving stimulus. For example, microsaccades are detrimental to detecting veridical motion50 and can induce motion-illusions51.

To preview our novel findings: (1) task difficulty delayed microsaccade rebounds and both delayed and increased the peak of task-evoked pupillary responses; (2) microsaccade rebound times and peak pupillary responses correlated with discrimination response times; and (3) microsaccade rebound times correlated with pupillary peak responses across difficulty levels and observers. The joint delay in these oculomotor measures, which likely reflect a prolonged period of sensory evidence accumulation, support a shared neural mechanism underlying both pupil and microsaccade responses.

Methods

Experimental design

We analyzed psychophysical and eye data from a motion direction discrimination task in a recent study in which we measured performance for 8 “standard” drift directions at 8 polar angle locations of the visual field52. Both the drift directions and locations ranged from 0 to 315° in 45° increments (Fig. 1a), resulting in 64 unique direction-location conditions. Trial difficulty was defined in two ways – the standard direction used for discrimination and the tilt offset relative to the standard (Fig. 1b) – detailed after the experiment description. We chose these two metrics because motion direction better explains the behavioral variability for this task compared to other features, e.g., location in the visual field52, and because tilt offset was the direct manipulation of motion direction discriminability.

Fig. 1
figure 1

Experimental Stimulus and Procedure. (a) On each trial observers judged motion direction relative to 1 of 8 standard directions presented at 1 of 8 polar angle locations. (b) Difficulty of the motion direction discrimination task varied based on the standard motion direction (cardinal vs. oblique) and the tilt offsets of the stimulus with respect to the standard (8° vs 0.5—4°). The standard motion direction remained constant throughout a given experimental session whereas the tilt offsets were varied randomly across trials within each session. (c) Trial sequence of the motion direction discrimination task. Observers (n = 8) responded to whether the Gabor stimulus was drifting clockwise or counterclockwise relative to the standard direction.

Full size image

Observers Eight observers participated in the experiment with normal or corrected-to-normal vision (7 females, 1 male). All but one observer (an author, S04) were naive to the purpose of the experiment. Two observers (S07 and S08) only participated in half of the experiment but completed an equal number of easy and hard trials with respect to both difficulty metrics. The experimental protocol was approved by the New York University Institutional Review Board (IRB). All procedures were conducted in accordance with the Declaration of Helsinki and relevant institutional guidelines and regulations. Written informed consent was obtained from all observers prior to their participation in the study.

Stimulus Sequence & Procedure Observers were instructed to discriminate the motion direction of a Gabor stimulus that drifted clockwise or counterclockwise relative to a standard direction. Observers reported whether the Gabor target’s drift direction was clockwise or counterclockwise relative to an internal standard direction, which was verbally communicated to the observer, and learned during a practice block prior to each experimental session. During the practice block, the target drift direction deviated ± 8° from the standard, and observers practiced until they reached ceiling accuracy. The standard direction was verbally communicated to the observer at the start of the practice block and remained constant throughout the session. Within each session, the stimulus appeared in one of four possible polar angle locations (either on the horizontal/vertical meridians or the intercardinal meridians of the visual field). Across the entire experiment, observers completed 12,800 trials across 8 polar angle locations.

The Gabor stimulus consisted of a 50% contrast 1-cpd square grating embedded in a stationary gaussian envelope with a standard deviation of 0.43°, truncated at 3 standard deviations from the center (diameter of ~ 2.5°). The stimulus drifted with a speed of 8°/s at 7° eccentricity. The angular offset between the stimulus drift and standard direction was determined by a method of constant stimuli (± 0.5–8° logarithmically spaced). Each of the 10 offsets included 20 repeats for a given polar angle location and motion direction. Observers completed the sessions in a dark room. The display was covered with matte black cardboard, except for a central circular aperture cutout. The distance between the eye and the monitor was 60 cm, and a chin and forehead rest (SR Research Ltd., Ontario, Canada) was used to ensure a properly aligned and stabilized head position. All stimuli were presented using Psychtoolbox and MATLAB.

Each trial began with a 1300 ms fixation period, followed by a 500 ms stimulus period (Fig. 1c). Immediately after the stimulus period, the fixation marker changed from black to red, prompting the observer to respond. Observers had unlimited time to indicate their response via key presses (right or left arrow keys for counterclockwise and clockwise respectively). Auditory feedback was given for correct (high tone) and incorrect (low tone) responses at the end of the trial.

Trial Difficulty Trial difficulty was assessed using two metrics. The first difficulty metric was based on the motion direction of the standard direction used for discrimination. Trials in which observers discriminated cardinal directions were considered easy; and oblique directions were considered hard (sensitivity is greater for cardinal than oblique53,54). The standard direction was set to either cardinal or oblique per experimental session, with order counterbalanced across participants. The second difficulty metric depended on the magnitude of the angular offset between the stimulus and standard direction; greater angular distances are easier to discriminate. Easy trials had a large tilt offset of ± 8° and hard trials had small tilt offsets between ± 0.5–4°. The angular offsets were ordered randomly across trials. Thus, at the start of each trial, observers did not know whether the stimulus would be easy/difficult to discriminate in terms of angular offset but they did know the standard motion direction (Fig. 1b).

Eye Data Collection Eye data for the dominant eye was collected using Eyelink 1000 Plus Desktop Mount Eyetracker (SR Research Ltd., Ontario, Canada), using the monocular tracking and pupil corneal reflection (pupil-CR) modes. Sampling rate was 1000 Hz (44% of data), 2000 Hz (42% of data, downsampled to 1000 Hz) or 500 Hz (14% of data, upsampled to 1000 Hz using pchip interpolation which preserves original values and shape of the data; we obtained the same results without interpolation). Fixation was enforced within 1.5 dva of a small circular fixation marker from 300 ms prior to the stimulus onset until the stimulus offset. Trials with broken fixation during this period were aborted and replaced at the end of the session. Approximately 2% of trials for S02 and 12.5% of trials for S07 were not recorded due to eye tracker error, so they were not included in the analysis. (~ 1.3% of the total trials).

Software The experiment was written using MATLAB 2022b (The MathWorks, Natick, MA) and the Psychophysics Toolbox extensions (Psychtoolbox)55,56. All analysis code was written using MATLAB 2022b (The MathWorks, Natick, MA). We used the PRET toolbox57,58 for pupil analysis.

Statistical analysis

Microsaccade Preprocessing & Detection Microsaccades were detected based on a standard velocity-based algorithm previously described1,59. This involved bandpass filtering the data with a Hamming window using filtfilt.m in MATLAB. A central distance method was used to compute the change in horizontal and vertical eye position from the smoothed X, Y data by deriving the distance of three consecutive samples for (t + 2—t-1) and (t + 1—t-2). The two resulting displacement values symmetric to t were averaged then multiplied by one third of the sampling rate to convert to velocity. To qualify as a microsaccade, eye movement velocity had to exceed a trial-specific elliptical threshold in 2D velocity space. This threshold was defined as six times the median-based standard deviation of horizontal and vertical eye position velocity within the trial59.

Candidate microsaccades were further constrained to have a minimum duration of 6 ms. The remaining microsaccades which occurred within 15 samples (ms) were merged to exclude segments resulting from overshoot. For each microsaccade, amplitude was computed as the distance from the eye position start to end. Amplitude and velocity were then converted from pixel units to degrees of visual angle. Microsaccades were constricted to have an amplitude between 0.05 and 1 degrees. The lower bound was determined based on the microsaccade resolution reported in the SR Research Eyelink 1000 Plus documentation (SR Research Ltd. 2016). The scale of a typical microsaccade in relation to the stimulus is represented in Fig. 2a. The main sequence of microsaccades reflects the typical relation between velocity and amplitude (Fig. 2b) and the correlation between these two was high for each observer (range = 0.85 to 0.98; mean = 0.91). Within trials, epochs with blinks or saccades were identified using built-in online detection algorithms of the Eyelink system. Additional saccades, identified as eye velocity epochs that fulfilled the velocity criteria used for microsaccade detection, but were greater than 1 degree in amplitude, were also filtered out of the data. Filtering out these epochs was necessary to accurately compute microsaccade rate across all trials.

Fig. 2
figure 2

Quantification of microsaccades. (a) Approximate scale of fixational eye movements (magenta: microsaccade, ocular drift: black) in relation to the fixation marker and the stimulus. (b) The main sequence for an example observer shows the expected linear relation between peak velocity and amplitude. Each data point represents a microsaccade across the entire experiment.

Full size image

Characterizing Microsaccade Rate, Inhibition, and Rebound Microsaccade rates were computed for each observer, experimental session, and for each difficulty condition, smoothed using a gaussian kernel with a sliding window of 100 ms and then averaged together. To evaluate whether certain timepoints in the trial period data differ significantly across conditions at the group level, we employed a nonparametric cluster-based permutation test60 for dependent measures. Cluster-based permutation tests are a common method to compare time series while controlling for multiple comparisons. This method tests for trial epochs with significantly different microsaccade rates. Note that this analysis is not sensitive to detecting phase shifts in the time series, or delays in canonical temporal characteristics of oculomotor behavior (e.g., microsaccade rebound), which is addressed below. Our cluster analysis included 8 paired trial averages for ‘easy’ versus ‘hard’ conditions (1 per observer), resulting in a total of 256 permutations. Cluster significance was determined by a 0.05 p-value threshold, with adjustments for multiple comparisons using the permutation distribution.

Next, we quantified the time at which the rebound from inhibition occurred from the microsaccade rate time series. This point of inflection could indicate when the observer had accumulated enough information and thus would occur later for more difficult trials. Inhibition was approximated as the time point between the stimulus onset and offset with the lowest microsaccade rate, determined using a sliding average of 250 ms. The rebound was computed as the time from inhibition to when the first microsaccade was 50% likely to have occurred using the temporal probability density function. Bootstrapped 95% confidence intervals were computed by resampling an equal number of trials per condition (with replacement), generating distributions for each density function. This method is better suited than the cluster analysis above for comparing delays in microsaccade rebound.

Preprocessing Pupil Data The raw pupil size data were epoched into segments on a trial wise basis. Due to the long, sluggish nature of pupillary responses, blinks were unlikely to affect the low frequency changes in pupil amplitude; thus, to produce a smooth time series of pupillary responses for trial wise model fitting, blink periods were reconstructed using cubic-spline interpolation61. The raw data was then bandpass filtered using the MATLAB function filtfilt.m with cutoff frequencies of 0.03 and 10 Hz. Pupil time series for each trial were then standardized relative to a within-trial baseline epoch; this epoch corresponded to when fixation was experimentally enforced (see Methods: Eye data collection). The time series were centered around the minimum value during this stable baseline epoch – this aligned pupil responses and corrected for drift across trials while preserving trial by trial amplitude modulations. We then divided the signal by the mean within the baseline period across the entire session. Finally, we scaled these values by the grand baseline mean across all trials and sessions over the grand range to reflect percent signal change.

Modeling Pupil Dynamics The time series of the pupil data were modeled with a general linear model (GLM) framework57,58. Pupil size was estimated on a trial wise basis, as a sum of dilatory responses for three neural events (impulses) and a boxcar function each convolved with a pupil response function (puRF). Each impulse had free parameters for amplitude (% signal change) and peak (ms), and represented the internal signals associated with the onset of several trial events. The convolved impulses were fit to the pupil responses evoked by each event, including: (1) the start of the trial, (2) the discrimination task, and (3) the observer’s response (examples in Fig. 3a). The timing of these events were estimated within specific temporal windows, which allowed for variable estimates in time for each respective internal signal: the impulse of the first event was constrained to occur prior to the stimulus onset; the second event within the period from stimulus onset to the observer’s response (which varied in time across trials); and the third event -500 to 1500 ms relative to the observer’s response.

Fig. 3
figure 3

Model of the pupil response. (a) The modeled neural events correspond to the (1) trial start, (2) stimulus-evoked onset, (3) response-evoked onset (as impulses in green), and a sustained neural event during the stimulus period as a boxcar function. The impulses could vary in time and amplitude, and the boxcar varies in amplitude but is temporally fixed to the stimulus period (0–500 ms indicated by the shaded blue region). (b) A neural event (green) and the pupillary response function (Erlang gamma function in gray). The dashed line indicates the time of peak, which is fixed to 930 ms from the neural event. (c) The data (blue) and the model fit (black dashed line) resulting from the estimated effects per event (green) and the boxcar (gray). The response time for the trial is shown as a red dashed line.

Full size image

The boxcar function, also convolved with the puRF (Fig. 3b), accounted for the sustained pupillary response resulting from the stimulus period. The boxcar, unlike the neural events, was fixed in time (stimulus onset to offset) but included a free parameter for amplitude. The puRF was modeled as an Erlang gamma function Γ(k,λ) with fixed parameters k = 10.1, and λ = 0.01 ms which set the puRF shape and temporal profile, respectively, and resulted in a fixed timing of 930 ms between any estimated neural event and its corresponding peak pupillary response. These values were chosen based on estimated parameters of the Erlang gamma function from several observers in a separate study62.

An optimization procedure using the PRET toolbox57,58 was used to estimate the free parameters per trial (4 amplitude + 3 neural event timings). Together, these reconstructed signals are summed to predict the pupil dynamics throughout each trial segment (Fig. 3c). The model was fit for each trial separately. We included trials with estimates that account for at least 50% of the variance of the actual pupillary response. This filtering ensured that parameter estimates adequately characterized the underlying data before using them as summary statistics, excluding trials (29,163 of 87,200 across all observers). A similar percentage of trials was included for easy and difficult tilts (64% vs 66%) and for cardinal and oblique (66% vs 66%) judgments. We then averaged parameter estimates per event and for the boxcar for each observer.

Observer Discriminability Discriminability was computed as d′ for each difficulty condition to confirm the difficulty manipulations were successful. We additionally computed d′ to compare discriminability for trials with and without microsaccades during the stimulus period. We binned the data to compare like-trials with the same direction and tilt offset (8 motion direction; 5 tilt offsets). This was done to evaluate whether the presence of microsaccades modulated performance differences as a function of difficulty. Additionally, we equated the number of trials between the “with microsaccade” and “without microsaccade” conditions by randomly selecting a subset of trials for the condition with greater trials. For all d′ calculations, hits were defined as correct responses to counterclockwise stimuli and false alarms were incorrect responses to clockwise stimuli. Adjustments to avoid potential infinite values involved a standard procedure which replaced rates of 1 with (n—0.5) / n, and rates of 0 were replaced with 0.5 / n, where n is the number of trials that were counterclockwise (for hit rate) or clockwise (for false alarm rate)63.

Statistical Tests Unless otherwise specified, statistical comparisons between easy and difficult measures were computed using two-tailed paired t-tests, with a significance threshold of p < 0.05. We also mention marginally significant trends (p < 0.1). Correlations were computed using Pearson’s r, and partial correlations were computed by first regressing out covariates (e.g., observer or difficulty) and then calculating the partial Pearson’s correlation rp on the residuals. All effect sizes were computed with Cohen’s d, where d = 0.2 indicates a small effect, d = 0.5 indicates a medium effect, and d = 0.8 indicates a large effect64.

Results

Task performance We first evaluated task performance by calculating differences in d′ and response time across all polar angle locations for each difficulty condition. Consistent with previous reports52,53,54, discriminability was better (t(7) = 3.728, p = 0.007, d = 1.318) for cardinal (mean d′ = 1.78) than oblique (mean d′ = 1.01) motion directions. Observers also responded faster (t(7) = -3.168, p = 0.016, d = -1.120) when discriminating cardinal (mean RT = 670 ms) than oblique (mean RT = 700 ms) directions (Fig. 4a-b). Discriminability for motion directions with large offsets from the standard direction exceeded (t(7) = 11.269, p = 9.6 × 10–6) directions with small offsets. Response times were also faster (t(7) = −2.532, p = 0.039, d = 0.895) for large (mean RT = 700 ms) than small (mean RT = 740 ms) offsets (Fig. 4c, d). Greater discriminability and faster response times for easy trials demonstrate that both difficulty manipulations were successful.

Fig. 4
figure 4

Performance on the motion direction discrimination task. (a) Discriminability (measured by d′) was significantly greater and (b) mean response times were lower for cardinal than oblique directions and (c-d) for large than small tilt offsets. Opaque dots represent the sample means; translucent dots represent each observer. Error bars reflect the standard error of the difference.

Full size image

Microsaccades Rates Observers showed a decrease in microsaccade rates prior to the stimulus onset, consistent with prestimulus inhibition for a temporally expected stimulus29,30. This inhibition was followed by an inflection point, with a subsequent increase in microsaccade rates typically between 200–300 ms after the maximal inhibition (see Fig. 5). The time at which inhibition and rebound occurs with respect to stimulus onset varied considerably across observers (see Supplementary Fig S1). At the group level, microsaccade rates were lower for oblique than cardinal directions from 430 to 614 ms relative to stimulus onset, as determined by a cluster-based permutation test (p = 0.012, 256 permutations, d = 1.483). Similarly, microsaccade rates were lower for small than large tilt offsets from 421 to 603 ms relative to the stimulus onset (p = 0.004, 256 permutations, d = 1.426). In summary, microsaccades were more suppressed for difficult trials; this difference in inhibition started prior to the stimulus offset and lasted for ~ 200 ms.

Fig. 5
figure 5

Median microsaccade rates across observers are lower for trials with greater difficulty. (a) microsaccade rate for trials with stimuli drifting in cardinal (green) and oblique (purple) directions. Stimulus period is represented in the shaded region: stimulus onset (time = 0 ms) and offset (time = 500 ms). The response cue occurs immediately after the stimulus disappears (time = 500 ms). (b) microsaccade rate for trials with stimuli with large (black) and small (gray) tilt offsets relative to the standard direction. Asterisks mark significant clusters of p ≤ 0.01.

Full size image

Microsaccade Rebound and Performance To further assess how temporal dynamics vary with difficulty, we next computed the rebound. Microsaccade rebound, defined by the time of the first post-inhibition microsaccade, is a more conservative measure than overall microsaccade rate reported in the previous section. First, the cumulative probability density was calculated per difficulty condition per observer (Fig. 6). Then, derived from the density distribution, the rebound was defined as the time relative to stimulus onset for which a microsaccade was 50% likely to have occurred. At the group level, rebounds occurred earlier for cardinal (M = 523 ms, SD = 100 ms) than oblique (M = 579 ms, SD = 146 ms) directions (t(7) =  − 2.822, p = 0.026, d = 0.998). For tilt magnitudes, there was a marginal trend between large (M = 520 ms, SD = 94 ms) and small (M = 558 ms, SD = 133 ms) tilt offsets (t(7) =  − 2.177, p = 0.067, d = 0.770).

Fig. 6
figure 6

Example observer (S02): Cumulative probability density function for the time of first microsaccade after inhibition. The rebound is delayed for difficult trials. (a) Probability density function for the first microsaccade after inhibition to occur for cardinal (green) and oblique (purple) motion direction discrimination. Confidence interval bands represent the 95% CI derived from bootstrapping. Dashed lines indicate the time since stimulus onset (in ms) for the post-inhibition microsaccade to have 50% likelihood to have already occurred. (b) Same plots for tilt offset; conventions as the left plot.

Full size image

We next assessed whether microsaccade rebound and discrimination response times were jointly delayed. We computed partial correlations because the variability of rebounds and response times can stem from differences in both observers and task difficulty. This was done separately for the two difficulty metrics–standard direction and tilt offset. For data binned by standard direction, discrimination response times were positively correlated with the time of microsaccade rebound (r(14) = 0.870, p = 1.2 × 10–5); and correlated across observers when controlling for the standard direction (rp(14) = 0.863, p = 1.7 × 10–5), and across difficulty measures, when controlling for observer differences (rp(14) = 0.908, p = 1.2 × 10–6) (Fig. 7a). Response times additionally correlated with rebounds for the alternative difficulty metric of tilt offset (r(14) = 0.830, p = 7.1 × 10–5); and correlated across observers when controlling for difficulty (rp(14) = 0.824, p = 8.9 × 10–5), and across difficulty when controlling for observer (rp(14) = 0.871, p = 1.2 × 10–5) (Fig. 7b). These findings indicate that a joint delay in rebound and response time depend on both the observer and task difficulty.

Fig. 7
figure 7

Microsaccade rebound is correlated with observers’ response time. (a) Left: data for each observer’s cardinal (green) and oblique (purple) trials. Center and right: Partial correlations while separately regressing out effects of standard direction and observer. Plotted line is the least squares regression fit to the data points. (b) same conventions as A, for large (black) and small (gray) offset trials.

Full size image

Having established that microsaccade inhibition is greater and prolonged for difficult trials, we next investigated whether and how the presence of microsaccades relates to observer discriminability. We compared trials with and without microsaccades during a critical period (i.e., the 500 ms period between stimulus onset to offset). To investigate whether microsaccades during the 500 ms stimulus period correspond to better or worse performance, we computed d′ while accounting for trial difficulty. For this comparison, there were no significant differences for trials with and without microsaccades: cardinal directions with large tilts (t(7) = 0.370, p = 0.723, d = 0.131), cardinal directions with small tilts (t(7) = −0.432, p = 0.678, d = −0.153), oblique directions with large tilts (t(7) = −0.384, p = 0.713, d = −0.136), and oblique directions with small tilts (t(7) = −0.176, p = 0.865, d = −0.062) (Fig. 8). Thus, although trial difficulty determined the temporal dynamics of oculomotor behavior, microsaccades during the stimulus period did not correspond to better or worse performance in our task. This suggests microsaccade rebounds occur after sensory evidence is accumulated and could explain why microsaccades provide neither a benefit nor a cost in our motion discrimination task.

Fig. 8
figure 8

Discriminability does not differ when a microsaccade occurs during the 500 ms stimulus period. Discriminability for cardinal directions with large tilt offsets (dark green), cardinal directions with small offsets (light green), oblique directions with large tilt offsets (dark purple), and oblique directions with small offsets (light purple). X-axis indicates whether the trials contain microsaccade during the stimulus period (“w” for with microsaccades and “w/o” for without microsaccades).

Full size image

Pupil Area Pupil area was computed as the amplitude of the estimated pupil responses corresponding to the events in our model. Three impulse events were convolved with a puRF: the start of the trial (trial-evoked event), the discrimination task (task-evoked event), and the observer’s response (response-evoked event); additionally, a sustained (boxcar) event corresponding to the stimulus period was also convolved with the puRF. The timing for each of the three impulse events was estimated using an optimization procedure, and the sustained response was fixed in time consistent with the stimulus duration.

For the task-evoked event, pupil size was larger for oblique than cardinal directions (t(7) = 2.464, p = 0.043, d = 0.922), but did not differ for small and large tilts (t(7) = 1.698, p = 0.133, d = 0.530) (Fig. 9a-b). Critically, the amplitude of the sustained response was larger for oblique than cardinal directions (t(7) = 3.416, p = 0.011, d = 1.198), and for the small than large tilts (t(7) = 3.174, p = 0.016, d = 1.244) (Fig. 9a-b). The other modeled events, trial-start and response time, did not evoke different responses based on difficulty for standard direction (trial start: t(7) = 1.273, p = 0.244, d = 0.350; response: t(7) = 1.570, p = 0.161, d = 0.503) or tilt (trial start: t(7) = 0.058, p = 0.956, d = 0.031; response: t(7) = 1.654, p = 0.142, d = 0.494). In sum, pupil size was larger with difficulty for the neural events that corresponded to the stimulus period and the task, but not for the observer’s response or in anticipation of the stimulus onset.

Fig. 9
figure 9

Average difference in pupil area for each observer. (a) Pupil area is larger for oblique than cardinal motion directions (b) and for smaller than larger tilt magnitudes. Opaque dots represent the average; transparent dots are data for each observer. Error bars are the standard error of the mean. Unfilled dots represent the amplitude of the stimulus boxcar for each observer which has fixed and sustained event timing to reflect the stimulus period.

Full size image

Peak Pupil Response for each Event The timepoint of peak pupil response to each event was computed from model estimates. The peak pupil response can serve as an indicator of processing speed, where a delayed peak may indicate longer processing periods. The task-evoked pupillary response peaked later for oblique than cardinal directions: (t(7) = 3.670, p = 0.008, d = 1.297) and for smaller rather than larger tilts (t(7) = 4.456, p = 0.003, d = 1.576) (Fig. 10). For the trial-evoked event, the oblique trials were marginally delayed compared to the cardinal standard direction (t(7) = 2.166, p = 0.067, d = 0.766), probably because this difficulty factor was blocked, but did not differ for tilt offset (large vs. small tilts: t(7) = -0.920, p = 0.38, d = -0.325). There was no difference in peak timing for the response-evoked pupil in either cardinal versus oblique blocks: t(7) = -0.033, p = 0.975, d = -0.012, or large vs. small tilts: t(7) = -0.629, p = 0.549, d = -0.223. The sustained response was not included in this analysis given that the boxcar component of our model had a fixed timing parameter.

Fig. 10
figure 10

The estimated peak of each event included in the PRET model. Task-evoked responses peak later for more difficult trials based on both: (a) oblique motion directions, and (b) small tilt offsets.

Full size image

Relation between Peak Pupil Response and Discrimination Response Time. We next assessed whether peak pupil dilation for the three trial events correlated with discrimination response times. First, we focused on the peak dilation for the stimulus-evoked pupillary response as this was predicted to vary with response time. For data binned by standard direction, discrimination response times were positively correlated with the time of peak dilation (r(14) = 0.812, p = 1.4 × 10–4). This was the case, across observers when controlling for the standard direction (rp(14) = 0.800, p = 2.0 × 10–4) as well as across difficulty measures, when controlling for observer differences (rp(14) = 0.891, p = 3.6 × 10–6) (Fig. 11a). Response times also correlated with the pupillary peak for this event using the alternative difficulty metric of tilt offset (r(14) = 0.804, p = 1.7 × 10–4), both across observers when controlling for difficulty (rp(14) = 0.803, p = 1.8 × 10–4) as well as across difficulty when controlling for observer (rp(14) = 0.751, p = 7.9 × 10–4) (Fig. 11b). Thus, like microsaccade rebounds, the timing of peak pupil dilation was delayed with discrimination response time.

Fig. 11
figure 11

Observers’ estimated peak dilation correlates with discrimination response time for (a) standard direction and (b) tilt offset. Figure follows the same conventions as Fig. 7.

Full size image

For the pupillary event corresponding to the trial start, response time and peak dilation did not have a significant relation, neither for cardinal vs. oblique (when controlling for difficulty (rp(14) = -0.120, p = 0.657); when controlling for observer (rp(14) = 0.358, p = 0.173), nor for large vs. small tilt offset (when controlling for difficulty (rp(14) = -0.066, p = 0.807); when controlling for observer (rp(14) = -0.265, p = 0.321).

Lastly, for the response-evoked pupillary event, response time and peak dilation had a slight correlation across observers when controlling for difficulty for standard direction (rp(14) = 0.512, p = 0.043) and a marginal correlation for tilt offset (rp(14) = 0.473, p = 0.065) indicating that observers may preserve this relation during the response period. Conversely, when controlling for observer within this event, there was no significant correlation for either standard direction (rp(14) = 0.353, p = 0.181) nor tilt offset (rp(14) = 0.064, p = 0.815).

Relation between Microsaccade Rebound and Peak Pupillary Response. As both microsaccade rebound and peak pupillary responses were delayed with difficulty, we assessed whether they were jointly delayed across observers. Observers’ microsaccade rebounds positively correlated with the time of the pupil’s peak dilation for the task-evoked response (r(14) = 0.619, p = 0.011 for standard direction (Fig. 12a); r(14) = 0.589, p = 0.017 for tilt offset (Fig. 12b). Again, there was a positive correlation across subjects when controlling for difficulty (rp(14) = 0.593, p = 0.015 for standard direction; rp(14) = 0.572, p = 0.021 for tilt offset), and across difficulty when controlling for observer (rp(14) = 0.891, p = 3.6 × 10–6 for standard direction, rp(14) = 0.674, p = 0.004 for tilt offset). These correlations indicate that both trial difficulty and observer contributes to this relation between microsaccade rebound and peak pupil response.

Fig. 12
figure 12

Observers’ estimated peak dilation correlates with time of microsaccade rebound for (a) standard direction and (b) tilt offset. Figure follows the same conventions as Fig. 7.

Full size image

Discussion

We used a simple motion direction discrimination task in the perifovea to investigate how task difficulty jointly delays microsaccades and pupil responses. This task is well suited to ask this question because small eye movements can disrupt the processing of brief moving stimuli50. Observed discriminability and reaction times confirmed difficulty varied for both motion direction (cardinal and oblique) and tilt offset (large and small). We found that with increasing difficulty, microsaccade inhibition was more pronounced and prolonged, and the subsequent rebound and accompanying task-evoked pupillary response was larger and delayed. Microsaccade rebound correlated with task-evoked pupil response, and both oculomotor responses correlated with discrimination reaction time. Importantly, these relations held when separately controlling for both observer and difficulty.

Perception and eye movements often go together (review65), but they can be dissociated66,67. Oculomotor dynamics correlated with response times, but discriminability was similar for trials with and without microsaccades when controlling for difficulty. Similarly, the presence of microsaccades does not affect overall discriminability or the effects of covert endogenous spatial68,69 or temporal35 attention. Thus, although microsaccades provide useful metrics, their presence or absence cannot replace performance measures. Equal discriminability may be surprising given the loss of sensitivity70 and temporal compression71 during microsaccades. However, the very brief decrements associated with a microsaccade (including the 40–50 ms before and after a microsaccade)50 may not cause a substantial interruption of sensory evidence accumulation during the 500 ms stimulus presentation time.

In the current study, inhibition from suppression was non-reflective, it was mediated by cognitive processes related to difficulty of motion discrimination. The magnitude and onset of rebounds were unrelated to stimulus properties known to modulate them, such as contrast3,21,23, and spatial frequency23, as these stimulus properties were equal across all trials. Additionally, rebound delays cannot be attributed to temporal expectation, sustained fixation, or the duration of the response period, as they were all constant across trials. These factors, along with the numerous trials per observer and difficulty level, are propitious to uncover the relation between microsaccades and task difficulty.

All observers exhibited “pre-target inhibition” prior to the predicted stimulus onset. Task difficulty strengthened inhibition for difficult trials, consistent with previous studies8,13,15. Importantly, we showed that a delayed microsaccade rebound towards the end of the stimulus partially explains the lower rate for difficult trials, making the temporal profile critical in differentiating how oculomotor behavior depends on cognitive demands. Interestingly, all observers had rebounds within 300–500 ms after the stimulus onset, a window that includes the 350 ms canonical rebound time during reflexive inhibition. For easy motion discrimination trials, rebound times were more similar to those occurring after a transient stimulus that does not involve a task. As difficulty increased, the timescale of inhibition was prolonged, and rebounds occurred later. Thus, various aspects of task engagement –task difficulty, stimulus frequency, temporal expectation, temporal attention– can concurrently modulate inhibition duration, which typically lasts < 350 ms10 but can reach ~ 1 s37 before returning to the 1–2 Hz baseline rate.

We consider task difficulty as a modulator that prolongs inhibition duration, delaying the rebound due to longer accrual of sensory evidence, as microsaccades interrupt this process in perceptual decisions4. Similar delays in rebound occur when viewing or listening to rare stimuli in a sequence37,38,72, but not when passively viewing the same stimuli37, which suggests prolonged engagement or sensory accumulation for rare stimuli. We found that rebounds were more delayed for oblique versus cardinal standard directions than for small versus large tilt offset, potentially because difficulty of the standard direction was blocked (i.e., constant across the session) whereas tilt offset was randomized within sessions.

Pupil response is delayed with difficulty and microsaccade rebound. Changes in pupil size are considered to indicate difficulty, as pupil size increases with task demands (review6), such as alerting and orienting73, difficulty in orientation discrimination47, and greater perceived numerosity74. Our results are consistent with these findings, as difficult trials result in a larger pupil size for the stimulus- and task-evoked events. Additionally, this study reveals that difficulty additionally results in delayed temporal dynamics for both microsaccades and pupil responses, and corroborates that pupil and saccadic responses are time-locked and support a common neural source for both oculomotor responses47.

Prior findings have linked the amount of microsaccade inhibition and pupil size showing greater inhibition and larger pupil responses when preparing antisaccades (which the authors interpret as greater cognitive load) than prosaccades75. Our results are consistent with the notion that greater load amplifies the extent of inhibition and pupil dilation, and we further show difficulty delays both microsaccade rebounds and task-evoked pupil responses. These delays index the temporal processing windows for varying perceptual or cognitive load conditions, which are not captured by the magnitude of microsaccade inhibition or pupil size.

We found that the amount of microsaccade suppression does not depend on whether difficulty was blocked or intermixed, unlike prior work using pro- and antisaccades 75,76, as standard direction was blocked per session but tilt offset was intermixed within each block. Our protocol may be more suitable to investigate microsaccade effects that are not influenced by preparatory motor commands, as preparation of antisaccades can influence activity in the rostral superior colliculus77, a subcortical area linked to microsaccade generation.

Possible Physiological Correlates of Microsaccade and Pupil Response Effects We postulate that task difficulty modulations on oculomotor responses are mediated by the superior colliculus (SC), in agreement with prior proposals7, given its role in microsaccade generation78 and pupil modulation79. These SC modulations likely originate from high-level cognitive areas such as the dorsolateral prefrontal cortex (DLPFC) and anterior cingulate cortex (ACC), linked to task preparation and demand/difficulty, respectively80. Specifically, when these cortical areas are engaged in task-related processing, oculomotor responses are suppressed, likely by a reduction of SC activity. Increasing task difficulty, engages these cortical areas for longer periods, and leads to prolonged SC reduction (i.e., delayed microsaccade rebounds). Supporting this, signals from DLPFC81,82 and ACC82 modulate activity in SC directly; and indirect SC connections involve inhibitory basal ganglia83 and excitatory frontal eye fields84 projections.

The reduction of SC activity and its link to task difficulty is based on prior work implicating this area in oculomotor suppression. For example, greater SC suppression or enhanced omnipause neurons (OPN) activity can result in a brief reduction of motor signals that interfere with sensory input85,86, and lower perceptual thresholds87. Suppression of the foveal portion of SC is linked to microsaccade inhibition as activity of these neurons is greater before/during microsaccades and reversible inactivation decreases microsaccade rate78. Reflexive microsaccade inhibition, following a transient stimulus, results in a stimulus-evoked “interruption” of SC91 and a reduction of activity in the SC motor map3. Non-reflexive inhibition, mediated by cognitive engagement or task demands, could also reduce activity in the SC but at a different timescale. The intermediate and deep layers of SC integrate signals from visual, auditory, and somatosensory systems81, which might explain reduced microsaccade rates while expecting brief stimuli in vision29,30,31 as well as in other modalities32,33.

Greater microsaccade inhibition for difficult trials may result from a competition between information processing and motor commands; a similar competition has been considered for pupil responses88,89. Increasing demand would prolong competition and explain the observed later rebounds and discrimination response times. Such competition is reflected in other contexts (e.g., visual search), in which saccadic behavior adjusts with increasing task demands90. As perceptual decision-making requires bioenergetic resources, motor commands might be suppressed to decrease costs related to microsaccade execution33 and potential corollary discharge signals. The suppression of motor commands in SC would be released upon the formation of a perceptual decision, perhaps coinciding with transient bursts of SC activity that occur during microsaccade rebounds91. In this context, the temporal dynamics of oculomotor responses may depend on abrupt changes in neural activity–signaling task completion–within competing cortical regions that are implicated in motion direction discrimination. Such temporal dynamics of oculomotor behavior can thus uncover the timescale at which signals are relayed from the cortical network functionally and structurally connected to the SC.

Physiological models of microsaccade generation can help elucidate the physiological mechanisms explaining how difficulty modulates rebound from suppression, which would likely require cortical circuits that carry signals from decision-making areas. A potential model involves SC as the central hub determining whether a microsaccade is triggered, initiated when activity surpasses some threshold resulting from an integration of signals reflecting voluntary strategy and perceptual input92,93. Saccade-triggering has also been modeled as resulting from the dominance of activity from long-lead burst neurons over omnipause neurons94, and be mediated by inhibitory connections from the basal ganglia or excitatory connections from the frontal eye fields95. A potential expansion of these models could include parameters for task difficulty and/or the time of engagement.

Pupil size increments can serve as temporal indicators of perceptual decisions, for example, they happen just prior to reported perceptual rivalry transitions96. We find that task difficulty increases and delays peak pupil response, and the delay results from longer durations to form a perceptual decision. Importantly, the delay in the peak pupil response occurs only for the task-evoked event in our model–the event that directly corresponds to motion discrimination. The other trial events in our pupil model, related to trial onset and observer response, do not modulate with difficulty. This selective modulation suggests that pupil responses, though sluggish and delayed with respect to an event onset, are temporally linked to the online process of motion direction discrimination. Pupil size exhibited selective modulation for task and stimulus related events within the trial, complementing the temporal dynamics.

Increased pupil size can result from inhibited parasympathetic or activated sympathetic circuits controlling the constriction and dilatory iris muscles, respectively (review43); and both circuits receive input from the intermediate and layers of SC97. SC is also a candidate region that also time-locks the pupil response to fixational eye movements47,98, and microstimulation of its intermediate layers changes pupil size79. We speculate that as an observer shifts from accumulating sensory evidence to response preparation, the inhibition of the motor circuit transitions to excitation resulting in both microsaccade rate rebound and increased pupil size.

Conclusion This study revealed motor commands are inhibited during the perceptual decision-making process. This inhibition increases with task difficulty and correlates with response time. The microsaccade rebound terminates the motor inhibition and indicates a delayed transition to response preparation. Additionally, task difficulty results in larger and delayed pupil responses that are temporally linked to microsaccade rebound. Thus, the temporal dynamics of microsaccades can serve along with pupil dynamics as reliable on-line indicators of task difficulty.