Abstract
A reliable experience of time is critical for perception and action in the present, for accurately remembering our past, and for successfully planning a future. Theories of time perception commonly assume a central mechanism keeps time by providing a relatively independent, internal clock. Recent work, however, shows imaginary self-movements alter subjective time, suggesting a critical role for action in temporal cognition. To test the hypothesis that time perception derives from the relationship between action and perception, we designed a neuroadaptive interface operating on imaginary movement to visualize movement through virtual reality. EEG activity was classified online as reflecting accelerating movement or static imagery, which was then used in providing feedback for adapting the velocity of optical flow presented in a star field to enable neuroadaptive control. Two cybernetic experiments were conducted to determine how neuroadaptivity in the relation between action and perception affected temporal perception in the verbal time estimation task. In particularly, we contrasted neuroadaptive feedback (e.g. imagined running > visual acceleration) with non-adaptive (imagined standing > visual acceleration) and pseudoadaptive (sham) feedback conditions. Movement imagery biased estimated duration while intentional control increased judgements of the passage of time. We conclude that perception and imaginary action co-determine temporal cognition. Furthermore, the relationship between perception and action—our evaluation of perceived movement as intentionally produced—alters the subjective experience of time. Finally, we discuss the potential for our novel, neuroadaptive methodology as an investigative tool for temporal disturbances observed in psychopathology.
Similar content being viewed by others
Introduction
A reliable experience of time is critical to understanding our present position within our environment, connecting us to our past and permitting us to anticipate and control the future. A sense of time enables us to structure past events and distinguish these meaningfully from the present, suggesting a critical role for memory1. Furthermore, it allows us to make sense of the present moment, the fleeting glimpse of the here and now2, shaped by perception3,4, attention5, and working memory6. This sense of time is particularly critical for performance, as it enables us to act in time, for example keeping in synchrony with rhythms7 and allowing us to create music, alone or especially with others8,9,10. Finally, it permits a sense of the future, allowing stimulus and action anticipation, respectively the prediction of upcoming events to be perceived11 and acted upon12,13. The sense of time, in other words, is a critical cognitive function for executing control.
Indeed, disturbances in subjective timing have long been associated with a broad spectrum of psychological disorders14, although it has been difficult to determine the causal nature of this association. Thus, for example, patients suffering from depressive disorders commonly describe their experience of time as drawn out, passing by extremely slowly, causing frustration for keeping the patient ‘stuck in time’15,16,17. A variety of other psychiatric conditions, such as attention deficit hyperactivity disorder18,19, autism spectrum disorder20 and schizophrenia21, have likewise been described as “temporal syndromes”. However, while experiences of temporal disturbances can be diagnostic, it is unclear whether they are epiphenomenal to the disorders or by themselves contribute to their severity. To determine whether time perception plays a causative role, however, would require altering the sense of time and measuring the impact on psychopathology.
What kind of strategy would need to be deployed to alter the subjective sense of time depends on the theoretical assumptions of general time perception. In particular, one common assumption in theories of temporal cognition has been that differences in subjective time are predominantly due to the workings of a central mechanism. The best-known example of such a central theory of temporal cognition is the pacemaker-accumulator model22,23, which involves two parameters that can be adjusted to alter the sense of time. First, the pacemaker presents a hypothetical, relatively steady pace to function as an “internal clock”24, although this rate can be influenced by factors such as arousal25. Second, these ticks are collected in an accumulator whenever an attentional switch orients towards temporal judgement5. Strategies for altering the sense of time can therefore either focus on altering arousal—for example using relaxation strategies—or on modifying attention—for example by therapy to distract attention away from or towards time. However, the psychopathologies mentioned are themselves characterised by disturbances in arousal (or mood), attention (executive control) or both, making it difficult to ascertain whether there is any independent contribution of time perception.
In contrast to central theories of temporal cognition, action-perception integration theories26,27 generally understand cognition as driven by action-representation, and from this perspective follows an appreciation for affordances, motor processes, and the body itself in cognition. For example, from embodied cognition follows that time is not appraised in isolation, but is perceived necessarily in relationship to our body’s situation as a system28,29. Likewise, if, according to the sensorimotor theory of vision26, conscious awareness is the consequence of learning how motions result in sensory effects, it makes a certain sense to hypothesise that temporal awareness is caused by the dynamic interplay between motor execution and sensory perception. Whichever way we define the exact theory, however, such theories provide an intriguing alternative to traditional models for inspiring a strategy to modify the sense of subjective time: manipulate action to manifest temporal bias.
There is ample evidence that actions affect time perception, although it has taken long for the field to recognise such effects. Already in 1889, Hugo Münsterberg30 showed that motions such as breathing systematically affected temporal interval estimations (see also31), suggesting that the cyclic action of exhalation and inhalation was used to determine time. Recent work revived the idea that seemingly irrelevant movement can adjust time perception: Merely making irrelevant hand-movements was found to result in temporal compression (smaller subjective vs objective time) of visually presented intervals32. Interestingly, how such irrelevant actions are defined critically matters: the strength required to execute manual movements, the distance the hands travel, and even the orientation of the hands towards or away from us all bias temporal perception33,34,35,36. Both if we move (or if we cannot, c.f.37) and how we move therefore makes an impact on how we perceive time.
Indeed, it seems that even the mere thought of movement can bias subjective timing. Recently, we showed that not only does movement by itself affect time perception, but that even imaginary motion may do so38. Action simulation theory suggests imaginary movement is represented using the same neural substrates enabling actual movement39, explaining why it takes a similar amount of time to travel distance mentally as it does physically40,41. Thus assuming imaginary movements cause simulated motor activity, we38 showed imagining running faster and faster caused overestimation of a temporal interval of 7–16 s, while imagining walking slower and slower caused it to be underestimated. Imaginary action therefore not only affects action timing, but also the subjective sense of time in general. Of course, in non-imagined actions, movements typically result in congruent sensory feedback, so it remains unknown whether sensations caused by motor activity affects temporal cognition over and above perception and action independently.
In short, while the influence of both action—whether merely imaginary or motorically executed—and visual perception on subjective time are well-documented, it remains unclear whether the relationship between these areas of cognition has an independent contribution. To illustrate, consider forward movement in a car: these might be contingent on us pressing the pedal or releasing the brake, it might be related to someone else’s action if we take a passenger seat, or it might be seemingly random in a malfunctioning car. In other words, the contingency between action and perception may be defined as adaptive (phasic or anti-phasic), unrelated (with static action), or randomised (desynchronised or pseudoadaptive). Do these different relationships between perception and action, or sensorimotor contingencies, affect time perception over and above the effects of differences in stimuli and responses? To answer this question, we conducted two experiments in which imaginary actions were detected using a brain-computer interface and then used to adapt optical flow as presented in virtual reality.
Present study
To investigate whether the relationship between perception and action affects temporal cognition, we designed a novel, neuroadaptive42 device, which, as illustrated in Fig. 1, predicted movement imagery from brain activity and used this information to alter sensory feedback. A short training session was conducted to obtain ground-truth measurements for mental imagery of running and standing to train a machine-learning model for online classification of EEG activity. Two experiments were conducted to determine how neuroadaptive feedback alters subjective time. In both experiments, the effect of movement imagery adapting the optical flow within a star field presented in virtual reality was investigated. In the critical, neuroadaptive condition, running imagery was used to increase the velocity of the optic flow, while standing imagery reduced it. The task for participants was to estimate the duration of the star field’s presentation time (verbal estimation task,43) in seconds. As recent work in temporal cognition has shown dissociations between subjective durations (how much time has passed?), and velocity of time (How fast did time seem to go?)44,45, we furthermore requested participants to report their experience of the passage of time on a visual-analogue scale. In Experiment 1, we provided explicit imagery instructions to participants (see Methods, Experiment 1, procedure section) and investigated how mental imagery and neuroadaptive feedback together altered subjective time in the verbal estimation task. Experiment 2 used a free-choice paradigm instead, giving participants the sense of having free will to engage in imagery to investigate how the experience of control, as enabled by the neuroadaptive device, altered time experience.
For both experiments, effects of adaptivity and imagery were interpreted in accordance with predictions from central temporal cognition theory and perception-action theory. Central temporal cognition theories focus on perceptual and higher-level explanations of subjective time, and therefore would predict effects of adaptivity to the extent it increases optical flow, therefore predicting any non-static condition to increase duration estimation and passage of time judgements, regardless of the imagery task45. In contrast, perception-action and motor-centric theories of temporal cognition would predict effects of action, resulting in increased duration estimations for running imagery38. There is, of course, a distinct possibility that both theories hold, and that both optic flow and mental imagery have main effects. The more intriguing prediction from perception-action theory, therefore, would be if optic flow resulting from action imagery would have an interactive effect, with neuroadaptive conditions increasing the effect of optic flow on duration estimation and passage of time judgements.
Experimental control loop. Participant views star field in virtual reality (A), while imagining running or standing still (B). Brain activity corresponding to this activity is measured using EEG, which is bandpass (BP) filtered to provide spectral features across delta (\(\delta\): 2–4 Hz), theta (\(\theta\): 4–7 Hz), alpha (\(\alpha\): 8–12 Hz), beta (\(\beta\): 13–30 Hz), and gamma (\(\gamma\): 30–45 Hz) bands (C). Machine-learning allows online prediction of imagery to adapt the velocity of the star field, depending on the control condition (D).
Experiment 1
In Experiment 1, short intervals of moving stars were presented in virtual reality and the verbal estimation task46 was used to measure their subjective duration, while a judgement of the passage of time44 was used to measure the experience of time. Participants were instructed prior to these intervals to imagine either running faster and faster or standing still, while the movement of the stars was independently manipulated to be moving at a constant or adaptive, variable speed. Since the variability of the stars was controlled using the neuroadaptive device, the critical, neuroadaptive condition involved the combination of running imagery with adaptive star speed. As preregistered on aspredicted.org (https://aspredicted.org/h9bb5.pdf), we predicted an effect of running imagery on time estimation (replicating38), as well as an interaction between imagery and adaptivity. To identify whether the actual ability to control further affected time perception, the experiment furthermore included a pseudoadaptive condition, to directly compare neuroadaptive conditions with ”placebo” trials in which the star field was visually matched to a previous, recorded neuroadaptive trial, but not under control of the participant.
Results
To evaluate the neuroadaptive device itself, we analysed its accuracy in classifying training trials and evaluation scores by participants in terms of how well they were able to control the device. The cross-validation accuracy of calibration data was generally high, at 82.41% (SD = 9.58%). Parameterization of model algorithms was variable, with 30% of models using more restrictive (retaining 33% of features) and 70% less restrictive (retaining 50%) selections of features. 56.67% of models used a cost parameter of 10, 43.33% of 100. To determine the perceived effectiveness of the neuroadaptive device, a one-way repeated measures ANOVA was conducted on the control ratings with condition as factor, comparing control ratings across all five conditions (four experimental conditions and the pseudoadaptive condition). This showed a significant effect of condition, \(F (4, 112) = 25.86\), \(p <.00001\), \(\eta _{p}^{2} =.48\). As shown in Fig. 2, control was rated higher with the running/adaptive (\(63.17 \pm 5.88\)), pseudoadaptive (\(61.76 \pm 5.82\)), and standing/static (\(59.83 \pm 4.81\)) conditions than with the running/static (\(8.59 \pm 2.25\)), and standing/adaptive (\(32.90 \pm 5.88\)) conditions. In other words, control was rated high when visual feedback matched the imagery task. There was no significant difference between neuroadaptive (running/adaptive) and pseudoadaptive (control) conditions, \(t (28) = 0.27, \textit{p} =.79\).
Experienced control over the star field in different conditions. Participants were either asked to imagine standing still or running, and they were presented with a star field moving either at a constant, slow pace (static), or dynamically changing from slow to fast by adapting to movement imagery detection (adaptive). In pseudoadaptive conditions, participants were informed the star field would adapt to movement imagery detection, while in reality no true (online) adaptation occurred. Boxplots display distributions of individual average responses to the questionnaire item on experienced control (’How much control did you experience’).
Confirmatory analysis
In repeated measures ANOVAs with objective duration (8 vs 18 vs 30 s), imagery (running vs standing) and adaptivity (static vs adaptive) as factors showed objective duration was a clear factor in determining subjective duration, \(F (1.13, 31.61) = 242.14\), \(p < .00001\), \(\eta _{p}^{2} = .90\). Participants generally underestimated the objective durations by approximately \(30\%\) but clearly distinguished the intervals, with 8s estimated as \(6.1 \pm 0.38s\), 18s as \(12.59 \pm 0.79s\), and 30s as \(19.45 \pm 1.13 s\). Imagery significantly altered subjective duration, \(F (1, 28) = 13.37\), \(p = .001\), \(\eta _{p}^{2} = .32\), with running imagery adding \(1.29 \pm 0.35\)s to the estimates. Adaptivity had no significant main effect, nor entered into any significant interaction, \(\textit{p}s >.13\). Comparing neuroadaptive (running/adaptive) and pseudoadaptive conditions showed no significant difference in subjective duration, \(t (28) = 0.94\), \(p = .35\). The same analysis, but on passage of time judgements, showed a significant effect of objective duration, \(F (1.48, 41.13) = 32.30\), \(p <.00001\), \(\eta _{p}^{2} =.54\), and adaptivity, \(F (1, 28) = 33.14\), \(p <.00001\), \(\eta _{p}^{2} =.54\), but not imagery, \(p =.27\). Longer objective durations elicited slower passage of time judgements (8/18/30s : 54.44/46.71/42.49), while adaptivity resulted in enhanced passage of time judgements (56.30 vs static: 39.46). A significant interaction between objective duration and adaptivity, \(F (2, 56) = 5.50\), \(p = .007\), \(\eta _{p}^{2} =.16\), suggested this effect was more pronounced for the longer intervals (adaptivity effect at 8/18/30s: 13.12/18.34/19.08). Finally, a significant interaction between adaptivity and imagery, \(F (1, 28) = 12.30\), \(p = .002\), \(\eta _{p}^{2} =.31\), showed imagery to enhance the effect of adaptivity, as illustrated in Fig. 3. Comparing neuroadaptive with pseudoadaptive conditions showed no significant effect of control on passage of time judgements, \(t (28) = 0.77\), \(p = .44\).
Effects of movement imagery and adaptivity on duration estimations and passage of time judgements. Participants were either asked to imagine standing still or running while a star field was presented at either a constant, slow pace (static condition) or dynamically changing by adapting to movement imagery detection (adaptive). At the end of each trial, duration estimations (left) were entered in seconds, and passage of time judgements (right) were given by means of a visual-analogue scale. Error bars indicate within-subject standard errors.
Exploratory analysis
To evaluate the operation of the neuroadaptive device, we carried out an expansive repeated measures ANOVA on spectral power as measure with imagery (standing, running), adaptivity (static, adaptive), frequency (delta, theta, alpha, beta, gamma), and location (frontal, central, parietal, occipital) as factors and spectral power as measure showed all main effects were significant: imagery, \(F (1, 28) = 11.40\), \(p = .002\), adaptivity, \(F (1, 28) = 12.33\), \(p = .002\), frequency \(F (4, 112) = 306.61\), \(p < .0001\), and location, \(F (3, 84) = 20.12\), \(p < .0001\). Running \((3.96 \pm 0.07)\) induced higher power than standing \((3.91 \pm 0.07)\), and constant \((3.96 \pm 0.07)\) higher than adaptive \((3.92 \pm 0.07)\). Power was highest over occipital regions (\(4.30 \pm 0.11\)), followed by frontal (\(3.96 \pm 0.08\)), parietal (\(3.82 \pm 0.08\)), and central (\(3.67 \pm 0.07\)), and, as expected in EEG, highest for delta (\(4.97 \pm 0.07\)), followed by theta (\(4.56 \pm 0.09\)), alpha (\(4.38 \pm 0.11\)), beta (\(3.33 \pm 0.07\)), and gamma (\(2.44 \pm 0.07\)).
Imagery interacted with frequency, \(F (4, 112) = 22.65\), \(p < .0001\), with location, \(F (3, 84) = 17.84\), \(p < .0001\), and with the interaction between frequency and location, \(F (12, 336) = 4.49\), \(p < .0001\). Posthoc comparisons centring only on the differences in imagery conditions suggested its effect was largest over gamma bands (difference, \(D(run-stand) = 0.18 \pm 0.03\), \(p* < .0001\)), and beta bands (\(D = 0.05 \pm 0.02\), \(p* = .035\)), while all other frequencies had nonsignificant differences. It was mainly observed in occipital sites (\(D(run-stand) = 0.15 \pm 0.03\), \(p* = .0004\). The interaction effect suggested strongest differences in imagery were found in alpha/central (\(D = -0.14 \pm 0.04\), \(p* = .02\)), beta/occipital (\(D = 0.20 \pm 0.05\), \(p* = .003\)), gamma/frontal (\(D = 0.14 \pm 0.03\), \(p* = .0002\)), gamma/central (\(D = 0.11 \pm 0.03\), \(p* = .03\)), gamma/parietal (\(D = 0.12 \pm 0.02\), \(p* <.0001\)), and gamma occipital (\(D = 0.34 \pm 0.06\), \(p* < .0001\)). Adaptivity likewise interacted with frequency, \(F (4, 112) = 15.27\), \(p < .0001\), but not location, \(p = .64\), although it did enter a three-way interaction with location, \(F (12, 336) = 4.49\), \(p <.0001\). Posthoc comparisons centring only on the differences in adaptivity conditions suggested its effect was mainly significant in the alpha (Difference (D) \(adaptive-constant = -0.10 \pm 0.02\), \(p* = .002\)) and beta (\(D = -0.09 \pm 0.02\), \(p* = .0001\)) frequency bands. Within these frequency bands, it was observed in alpha/frontal (\(D = -0.08 \pm 0.02\), \(p* = .002\)), alpha/parietal (\(D = -0.14 \pm 0.03\), \(p* = .006\)), alpha/occipital (\(D = -0.13 \pm 0.03\), \(p* = .004\)), beta/frontal (\(D = -0.08 \pm 0.02\), \(p* = .002\)), beta/central (\(D = -0.08 \pm 0.02\), \(p* = .0002\)), beta/parietal (\(D = -0.10 \pm 0.02\), \(p* = .0002\)), and beta/occipital (\(D = -0.07 \pm 0.02\), \(p* = .03\)). Critically, however, imagery and adaptivity did not interact in any form, all ps \(>.38\).
A further analysis was carried out to determine if pseudoadaptive conditions differed from neuroadaptive conditions. This used a similar model to the normal model, but without the imagery factor, and with for adaptivity, the neuroadaptive in direct comparison to the pseudoadaptive condition. This showed the previous main factors of frequency, \(F (4, 122) = 275.49\), \(p < .0001\), and location \((F (3, 84) = 23.38\), \(p < .0001\)) to be similarly significant, but no difference was observed for adaptivity as either a main effect, \(F (1, 28) = 0.35\), \(p > .55\), or in interaction with frequency, location, or both, all Fs \(< 1.5\), all ps \(> .12\).
In summary, running imagery primarily increased gamma and beta power while adaptivity reduced alpha and beta power compared to constant conditions. These factors were relatively dissociable and did not interact in any way.
Conclusion
The present study aimed to determine how the ability to control optic flow through movement imagery affected time experience. A neuroadaptive device used machine learning on EEG activity to classify imagery movement types and alter visual movement online. Time experience was measured using two previously dissociated measures, of interval time estimation and judgements of the passage of time44,47. Behaviourally, we showed movement imagery increased estimated duration, replicating our previous, non-neuroadaptive work38. Furthermore replicating behavioural studies45, increased speed of optic flow was associated with faster experienced passage of time. More interestingly, mental imagery interacted with adaptivity only for passage of time judgements, with the fastest passage of time experienced for the neuroadaptive condition in which action (imagery) resulted in (optic flow) perception.
Since the neuroadaptive condition used machine learning to detect movement imagery from brain activity, it is not straightforward to determine the relationship between experimental conditions, brain activity, and time perception, but to obtain a better understanding of how the neuroadaptive device operated, we explored the induced effects of task conditions on the power spectrum of the EEG. This showed running imagery increased power in the gamma and beta frequency band, while adaptivity reduced activity in the alpha and beta band. While movement imagery is commonly associated with the beta rhythm, it may be surprising that running shows increased activity rather than the suppression normally observed against resting conditions48. However, note that standing conditions are not without imagery, but as per instruction involved somatosensory imagery, for which reason the difference in conditions may rather reflect a qualitative than quantitative change in imagery (c.f.49,50). More expected is the effect of adaptivity as reducing alpha power, as both adaptive/running and adaptive/standing conditions involve more visual engagement in the dynamic changes in optic flow, which is commonly associated with alpha power suppression51,52. The main result of the exploratory analysis was a consistent lack of interaction between adaptivity and imagery, which may be due to a variety of reasons, or suggest the general changes in the power spectrum are more similar to the changes in duration estimation (in which also no interaction was observed) than to the effects on the passage of time judgements.
Indeed, for the behavioural results, imagination interacted with adaptive optic flow for passage of time judgements, with the neuroadaptive condition producing the fastest passage of time experience. Thus, the results of experiment 1 suggest that a neuroadaptive device may alter the subjective experience of time. Specifically, the device manifests perception of optic flow by online detecting of action imagery, potentially strengthening the connection between imagination and sensory contingencies, inducing an experience of intentional control.
However, it remains unclear whether this effect was due to the ability to exercise control, or due to the mere illusion of control: The pseudoadaptive and neuroadaptive conditions may have been indistinguishable, which could mean illusory control is sufficient for altered time perception. Experiment 1, however, did not provide strong opportunity for introspectively evaluating control: Reports of perceived control were only obtained during the training, not the testing trials. Furthermore, the task of estimating control over a completely new modality is a complicated one for which participants received little time—at most 30 seconds. We designed a follow-up experiment to enable participants to better estimate control.
Experiment 2
According to ideomotor theory, we perceive being in control by observing the expected sensory consequences of intentional actions53,54, yet Experiment 1 provided little opportunity for participants to obtain any diagnostic information to dissociate real from illusory control. Consider, for example, a participant who is tasked with imagining themselves running and sees the stars speeding up in accordance with the neuroadaptive condition. They would naturally infer being in control. Pseudoadaptive conditions, however, play back the visuals based on previously recorded imagery. That means, if the participant correctly followed the instruction during the recorded trial, they should have responded exactly then as expected now, resulting in illusory sense of control. Neuroadaptive and pseudoadaptive conditions were thus designed to match one another precisely apart from the temporal synchrony between mental imagery and action perception. For example, if in the neuroadaptive condition, the running imagery started after 3 s, but in the pseudoadaptive condition only after 5 seconds, the recorded condition should show the star field accelerating before the action imagery commenced. The forced-choice paradigm and short duration of intervals therefore both contribute to there being very few moments of potential asynchrony between intentional action and perception, limiting the accumulation of evidence for judging control.
To allow more diagnostic information, we designed Experiment 2 as a free-choice time perception experiment with long interval estimations. The free-choice paradigm enabled participants to freely choose, within a trial, when to increase the star field velocity by imagining running faster and faster and when to decrease it by imagining standing still (see Methods, Experiment 2, procedure section). Since switching was encouraged by instruction to occur frequently and flexibly, certain moments within pseudoadaptive trials were expected to involve incongruence between action imagery and perception. Therefore, participants would have the critical information to introspectively estimate control, enabling us to determine how control perception affected subjective time experience.
Results
Confirmatory analysis
Three 1-way repeated measures ANOVAs were conducted with condition (neuroadaptive, static, pseudoadaptive) as factor and control, estimated duration, and passage of time judgement as dependents. These showed a significant effect of condition on control, \(F (2, 56) = 19.82\), \(p < .0001\), \(\eta _{p}^{2} = .41\), as well as on passage of time, \(F (2, 56) = 32.29\), \(p < .0001\), \(\eta _{p}^{2} = .54\), but not on estimated duration, \(F (2, 56) = 1.09\), \(p = .34\), \(\eta _{p}^{2} =.037\). Neuroadaptive (\(47.03 \pm 4.67\)) and pseudoadaptive (\(36.14 \pm 4.37\)) both elicited higher control ratings, \(\textit{p}s <.0002\), than static conditions (\(10.59 \pm 3.75\)), although no significant difference was observed between neuroadaptive and pseudoadaptive conditions, \(t (28) = 1.89\), \(p = .069\) (see Fig. 4). Likewise, in both neuroadaptive (\(45.10 \pm 4.41\)) and pseudoadaptive (\(42.28 \pm 4.13\)) conditions time was judged to pass faster than in the static (\(17.03 \pm 2.38\)) conditions, \(\textit{p}s <.0001\). The difference between the neuroadaptive and pseudoadaptive conditions was not significant, \(t (28) = 0.71\), \(p = .48\).
Experienced control, duration estimations, and passage of time judgements in different conditions. Participants were asked to freely engage in standing still or running, while being presented with a star field moving either at a constant, slow pace (static), or dynamically changing from slow to fast, adapting to actual (adaptive) or fake (pseudoadaptive) movement imagery detection. Boxplots for experience of control display distributions of individual average responses to the questionnaire item on experienced control (’How much control did you experience’). The middle panel shows the average estimated durations of the five minute trials, and the right panel displays passage of time judgements on a visual analogue scale. Error bars display standard errors of means. .
Exploratory analyses
Two linear mixed models were used to further investigate the effect of control in predicting estimated duration and passage of time. A linear mixed model was run, which in addition to the condition factor that was reported in the preceding analysis furthermore the covariates of optic flow (the average velocity of the star field) and control (reported on a visual analogue scale from 0 to 100). Since the condition was expected to affect control, we furthermore added the interaction between \(condition \times control\) to the model. This showed for duration estimations no significant effects on a small, significant effects in any main factor or in the interaction, all Fs \(< 2.4\), all ps \(> 0.13\). In contrast, however, passage of time judgements were now only predicted by differences in control, while the previously reported effects of condition were no longer significant. For easy comparison between the basic model with only the experimental condition, and the more complex multilevel model, please see Table 1, which compares directly these two models along with a medium complexity model, which includes both condition and optic flow. Higher reported control generally resulted in faster passage of time judgements, with an increase of 1 point of control (on a scale of 0 to 100) resulting in 0.32 points in passage of time (see also Fig. 5). In other words, the differences in experiencing the passage of time differed between the conditions not due to the simple differences in conditions, but due to the effect these conditions had on experiences of control.
Experienced control and passage of time in static, pseudoadaptive, and neuroadaptive conditions..
In repeated measures ANOVA with adaptivity (neuroadaptive, pseudoadaptive, static), frequency (delta, theta, alpha, beta, gamma), and location (frontal, central, parietal, occipital) as factors and spectral power as measure, adaptivity, \(F (2, 56) = 6.53\), \(p = .003\), frequency, \(F (4, 112) = 291.85\), \(p < .0001\), and location, \(F (3, 84) = 22.59\), \(p < .0001\). These were similar to experiment 1: higher power induced by static than neuroadaptive, \(D = 0.09 \pm 0.03\), \(p* = .02\), or pseudoadaptive, \(D = 0.07 \pm 0.03\), \(p* = .03\), conditions (but no differences between neuroadaptive and pseudoadaptive conditions, \(D = 0.01\), \(t(28) = 0.62\), \(p = .54\)), lower spectral power for higher frequencies (highest at delta, lowest at gamma), and highest spectral power over occipital and frontal sites. Adaptivity also interacted with frequency, \(F (8, 224) = 2.18\), \(p = .03\), and with frequency-by-location, \(F (24, 672) = 2.11\), \(p = .002\). Differences between the three conditions were stronger for alpha (neuroadaptive: \(4.32 \pm 0.12\), pseudoadaptive: \(4.33 +- 0.12\), static: \(4.48 \pm 0.13\)), beta (\(3.27 \pm 0.09\), \(3.29 \pm 0.08\), \(3.41 +- 0.08\)), and gamma (\(2.48 \pm 0.11\), \(2.49 +- 0.10\), \(2.58 \pm 0.09\)) than delta (\(4.97 \pm 0.08\), \(4.99 \pm 0.08\), \(4.99 \pm 0.09\)) and theta (\(4.56 \pm 0.11\),, \(4.58 \pm 0.11\), \(4.60 \pm 0.12\)). To avoid redundant post-hoc analyses, we followed this analysis with a planned comparison focussing solely on the difference between pseudoadaptive and neuroadaptive conditions, dropping static conditions from analysis. This showed that both the interaction between adaptivity and location, \(F (3, 84) = 4.31\), \(p = .007\), and between adaptivity, location, and frequency, \(F (12, 336) = 3.46\), \(p < .0001\), remained significant, showing the results were not due to the visual effects of optic flow. The differences between neuroadaptive and pseudoadaptive were strongest over central sites, (Difference \((D) neuroadaptive-pseudoadaptive = -0.05 \pm 0.02\), \(p = .047\)), and in delta over central sites, \(t (28) = 2.36\), \(p = .03\), but neither of these differences would be significant when adjusting for multiple comparisons (\(p* =.08\), and \(p* =.60\) respectively). Since this suggests there is no clear single space/frequency in which neuroadaptive conditions differ from pseudoadaptive ones, we ran five two-way repeated measures ANOVAs (Bonferroni-adjusted) with adaptivity and location for each frequency band. This showed no interaction between location and adaptivity in delta, \(p = .48, theta\), \(p = .91\), or alpha, \(p = .06\), but significant interactions in beta, \(p = .006\), \(p* = .034\), and gamma, \(p = .009\), \(p* = .048\). The common pattern of the latter two interactions suggest a redistribution of oscillatory activity with increased spectral power over frontal and central sites for pseudoadaptive conditions, while power increased over occipital sites for neuroadaptive conditions.
In summary, replicating Experiment 1, we showed neuroadaptive and pseudoadaptive conditions reduced spectral power particularly in alpha, beta, and gamma frequencies. Further analyses directly contrasting neuroadaptive and pseudoadaptive conditions did not reveal a clearly defined difference in location and frequency, but a three-way interaction between adaptivity, location, and frequency suggested the conditions had spatial redistributions particularly for the beta and gamma frequency bands.
Conclusion
Experiment 2 was conducted to determine how perceived control on optical flow, as enabled by a mental imagery-driven neuroadaptive device, altered subjective time experience. The efficacy of the neuroadaptive device was investigated by contrasting the real, neuroadaptive control condition with the placebo, pseudoadaptive and static, no-control conditions. Duration estimation, judgement of time perception, and experienced control were evaluated using a free-choice long-interval verbal duration estimation task. Unlike in Experiment 1, no effects of condition on duration estimation were found. This was not unexpected since imagery could equally be applied across conditions due to the free-choice nature of the task. Furthermore, replicating Experiment 1, the passage of time judgements were shown to be enhanced relative to static conditions for both neuroadaptive and pseudoadaptive conditions. Interestingly, however, by adding self-reported control as a covariate, the differences between conditions were removed, indicating that the effect on time perception, in terms of the experience of the passage of time, were due to perceived control, and not due to the visual effects of motion. Finally, we partially replicated the effects of adaptivity on oscillatory activity in the EEG, with neuroadaptive conditions reducing alpha, beta, and gamma activity compared to static conditions, and the higher frequency bands showing some dissociations between the neuroadaptive experimental and pseudoadaptive control conditions.
Discussion
The present study demonstrates a novel technique for altering the subjective perception of time by enabling neuroadaptive control over optical flow by imagining movement. Previous work suggested imaginary movement increases subjective duration while optical flow increases passage of time judgements38,45. Here, we replicate these findings, but moreover provide evidence for a unique contribution of intentional control to subjective perception of time. In Experiment 1, we demonstrated that maximally enhanced passage of time is observed when optical flow is contingent on imagery. However, the short trials and imagery task setup in Experiment 1 provided insufficient opportunity for participants to truly experience control, so Experiment 2 was designed with longer trials and a paradigm in which participants could freely alternate their imagery. The findings showed perception of control better predicted passage of time than the mere presence of optic flow, suggesting that the evaluation of perception as contingent on action alters time perception. To further explain how this works, we will discuss neuroadaptivity from a design point of view, followed by an examination of its neurophysiological effects, and concluding by a discussion of its cognitive implications towards the role of action in time perception.
Substantial levels of perceived control were achieved using a neuroadaptive device that enabled a type of ideomotor control: executing motions by anticipating their sensory consequences26,55,56,57. That is, it used machine learning to classify mental imagery of running or standing from EEG. By providing intention-congruent feedback in terms of accelerating or decelerating optic flow, the device does more than provide neurofeedback, but rather provides a novel way of controlled interaction with the environment. By necessity, this type of design is cybernetic in nature: it involves a closed loop in which intentional imagery affects visual consequences, which in turn affects a participant’s perceived control, which will alter their further interaction with the device. This complicates the degree to which we can causally explain the relationship between neural activity and cognition, although comparisons on the degree to which EEG frequencies contribute to observed effects has previously shown ample utility58.
Accordingly, we compared for Experiment 1 the power spectrum of the EEG under different task conditions so as to obtain more information as to how the ”black box” of machine learning works and how mental imagery affects brain activity. Mental imagery was found to increase gamma power while adaptivity reduced alpha and beta power. The effect of mental imagery was surprising as the type of motor imagery involved in imagining running is more commonly associated with beta oscillations59,60, for which reason it is typically targeted by brain-computer interfaces that classify motor imagery to execute actions61,62. Of course, the present study was unlike a classic directional motor execution task or contrasted motor imagery with a lack of motor imagery: standing is more a different type of action than a complete inaction. Alternatively, the running condition may have encouraged the participants to imagine the sensorimotor sensations resulting from the condition (see Experiment 1: Procedure). It is plausible that this induced engagement of tactile spatial attention, which has previously been related to increased gamma49. The effects of adaptivity are difficult to interpret as they are necessarily confounded with effects of visual optic flow, but the main effects observed were in the alpha and beta ranges. This may be due to various higher order processes affected by the adaptivity, but the most straightforward explanation is that static conditions resulted in alpha oscillations63 as the lack of external stimulation resulted in attentional disengagement and reorienting attention internally51,64.
Since mental imagery and adaptivity did not show the predicted interaction in Experiment 1, and since no difference with a pseudoadaptive (placebo) condition was observed, we conducted Experiment 2, which had longer trials and free choice in imagery to allow participants time to evaluate their sense of control. While the free-choice paradigm made it difficult to identify precisely which effects were related to imagery, and which to visual effects, broad changes were replicated in the alpha, beta, and gamma frequency bands. Furthermore, and unlike Experiment 1, we found differences when directly comparing neuroadaptive with pseudoadaptive conditions, suggesting intentional, true control differed from visually matched, ”fake” control specifically for the beta and gamma frequency bands, with the former showing a redistribution of power towards occipital sites. These results may well remind the reader of several previously observed associations between beta oscillations and motor imagery, as well as manual control65,66, and motor intentions67. The gamma oscillations, on the other hand, echo previous suggestions of relations with higher-order functions, such as conscious perception68, object binding69, multisensory binding70, and perception-action binding71. However, the present study did not make specific EEG predictions, and with the cybernetic design limiting the degree to which causal explanations can be made, we believe these speculations require replication and confirmation before understanding their importance.
Indeed, the central topic of the study concerned the relation between perception-action and subjective time, and from a cognitive, functionalist perspective, the findings further underline the importance of action as fundamental to time perception. Indeed, central theories of temporal cognition, such as the pacemaker-accumulator type models of time perception essentially argue timing to be a consequence of a central mechanism, driven by perception, attention, and affective states22,23,25,72. In contrast, sensorimotor, embodied cognition, and perception/action integration theories view timing not merely as a perceptual instrument, but as a consequence of action and the body, theorizing that abilities and constraints to performance should affect time perception73,74. Continuing this line of work, we showed that movement imagery, which is assumed to result in simulated motor activity39,40, increases subjective duration estimations38. Furthermore, if such movement imagery results in optical flow, control is experienced, which results in increased experienced passage of time. These findings echo the intentional binding literature, in which timing of sensory outcomes are adjusted if they are contingent on voluntary action75,76,77. However, here we provide evidence that general, interval timing is likewise adjusted if visual properties are dynamically controlled by action. Interestingly, the degree to which actual control was exercised over optical flow was relatively unimportant: feelings of control best predicted passage of time experience, even when pseudoadaptive (simulated control) was misjudged as neuroadaptive (real) control. In other words, neither visual perception, nor imaginary motor execution, fully explains time perception without taking into account control: the degree to which we believe visual perceptions are contingent on intentional action.
While the study was inspired by the potential of action and action-induced perception to alter time perception, we cannot completely rule out indirect effects of experimental conditions on parameters involved in traditional models of time perception. First, it has been well demonstrated that attention, specifically towards timing, alters time perception5,78, and it may well be that attention was affected by the neuroadaptive device. However, if we view either imagery or imagery-controlled optical flow as a dual task, it should mainly incur attentional deficits, and therefore if anything should decrease estimates, not increase them. Still, imaginary running, if requiring effort akin to actual running79, might increase duration estimates in the same way it alters action timing80, although the literature suggests mental effort may also reduce accuracy81 or decrease them if the experience is perceived as positive. Of course, we might speculate that the neuroadaptive, flow-controlling conditions were generally the more interesting ones for participants, with static and seemingly randomly changing star fields resulting in general distractions. Such a layperson interpretation of attention, as in motivating or interesting, is perhaps better viewed in its relation with enjoyment, which may independently increase the pacemaker’s speed82 or do so via induced flow83.
Could the neuroadaptive conditions have resulted in enjoyment while other conditions induced frustration or boredom? Indeed, studies have shown that positive affect, particularly arousing positive affect, results in temporal biases typically summarised by the cliché that ”time flies when you’re having fun”25,84,85. Here, however, imagery and neuroadaptive conditions resulted in relative temporal overestimation rather than underestimation; if anything, ”time dragged”. On the other hand, neuroadaptive control was associated with speeding up of the feeling of time passing by, which is more consistent with the suggestion of enjoyment-induced fleetness of time. This seeming dissociation seems hard to explain, although judgements of the passage of time remain a relatively recent measure in the literature of time perception, so it is relatively unclear how it relates to estimates of the pacemaker’s rate44,47. In the meantime, it is possibly difficult to dissociate action-contingent perception from action-independent perception as typically the former are viewed as more relevant, satisfying, and enjoyable. Indeed, continuously perceiving intentional control over the environment within a seamless loop is essential to having ”flow experiences”, which is characterized by altered time perception86.
In sum, a cybernetic experiment showed imaginary movement, perceiving optical flow, and the ability—whether actual or merely perceived—to exercise control over optical flow combine to alter the sense of time. Studies of time perception provide accumulating evidence that not only perception and attention, but also action execution73 and imagery38 can bias subjective time. However, the present study bridges the gap between perceptual and motor accounts, showing the ability to manifest perceptual outcomes using motor imagery independently contributes to sensing the passage of time. This ability, however, is not relying on a homuncular ”free-willing” agent, as merely the illusion of control was shown to predict the temporal effects.
Methods
Experiment 1
Participants
A sample of N=31 volunteers was recruited for participating in the study. Data from two participants were partially lost due to technical errors and excluded from analysis. Of the remaining, eleven reported their gender as female, seventeen as male, and one as other, and their age was between 19 and 43 (M = 28.8, SD = 6.6) years. The experiment was conducted in accordance with the guidelines as set out by the Declaration of Helsinki. Participants received full instructions regarding the experimental procedure and were made aware of their rights, including the right to withdraw from the study at any point without fearing negative consequences. The study conformed to the guidelines laid out by the Declaration of Helsinki and the institutional review board. The study protocol has been reviewed and approved by the The University of Helsinki Ethical Committee in Social and Behavioural Sciences (statement 42/2021).
Stimuli and apparatus
The timed stimulus was a star field rendered as up to 3,000 white, spherical particles in Unity 3D, editor 2019.2.2f1 (Unity Technologies, San Francisco, CA) with a scale of 1 Unity virtual metric, moving in the direction of the participant. That is, a virtual camera location positioned as the participant’s viewpoint was oriented towards the star field’s directional origin after requesting the participant to look straight ahead while wearing the head-mounted display. During the experiment, the velocity of the star field was altered in nine levels from 0.1 to 15 (see neuroadaptivity), exponentially scaled so the subjective difference between any two levels was experienced similar. The experimental setup comprised two PCs—one for stimulus presentation, the other for EEG data processing—an Oculus Rift CV1 (Oculus VR, California, US) for virtual reality display, and a PC keyboard for response collection. The stimulus PC was an MSI gaming laptop with nVidia GTX 1080 graphics card running custom experimental software developed in Unity3D. This software (1) implemented the experimental design and trial procedure detailed below; (2) synchronized VR presentation with EEG recording; and (3) adapted VR presentation based on online EEG classification. The recording PC stored digitised EEG data, performed machine learning to detect mental imagery, as implemented in OpenViBE 3.0.087, and sent classifier triggers to the stimulus PC over a local area network. EEG was recorded from 32 electrodes placed on equidistant locations on an elastic cap (EasyCap) and amplified/digitised using a Brain Products QuickAmp 32 USB amplifier (Brain Products GmbH, Gilching, Germany).
Procedure
As illustrated in Fig. 6, the experiment was conducted in two phases, first, a calibration of the machine learning models and second, the experiment of testing the time estimation task. Following reading of instructions, signing of informed consent, and setting up of equipment, participants commenced the calibration trials for training the machine learning models. They were instructed to either imagine standing still or running faster and faster. Specifically, in the former case, they were instructed to imagine themselves standing while seeing things moving around them, for example as one might feel while being on a bus. For the latter case, they were instructed to imagine running, for example, as one might to catch a bus. They were furthermore instructed to avoid making any physical movements during mental imagery, and to start imagining as soon as either a running person pictogram or a standing person pictogram appeared. During the calibration phase, 20 running and 20 standing trials were presented in random order. Each trial started with the pictogram cue appearing for 0.5 s, followed by the star field as used during the experiment, slowly moving at its lowest velocity (1). After 9 s, a stop sign appeared for 0.5 s, and after an inter-trial interval of 1.5 s, the subsequent calibration trial was run.
Trial procedure of Experiment 1. The participants first performed the calibration phase of the experiment (left) before the real experiment (right) was conducted. Only in the testing part of the experiment was the star field’s velocity potentially under influence of mental movement imagery.
Following the calibration training procedure (ca 10 min) and optimization of the machine learning models (4 min), the system was informally tested for its ability to detect imagery movement online and update star field’s velocity accordingly. An online demonstration was conducted to verify correct operation. Here, participants were verbally instructed by the lab assistant to imagine running for a few seconds, then standing still to experience control over the star field. Next, an inverse inference test was conducted: the participant was asked to freely imagine either running or standing, and the lab assistant was asked to determine visually what they were imagining, e.g. if the stars appeared to go faster, she would say they were imagining running. If participants expressed the ability to control star fields or the lab assistant correctly guessed based on the visuals, the experiment was started. Should this informal test be failed, an additional, shorter (20 trials), calibration training sessions was conducted to increase accuracy. The complete setup of equipment and calibration took ca. 60 min on average.
The experiment of time estimation on imagery movement itself comprised 100 trials including 10 training trials and 90 experimental trials. As illustrated in the right side of figure, each trial cued the mental imagery to be performed by displaying a pictogram of a standing person or running person. Once participants pressed the down key, a blank screen was shown for 1 s before the star field appeared. Now, participants were expected to perform the mental imagery instructed by the pictogram for as long as the star field was displayed, which was for either 8, 18, or 30 seconds (following38) 1 s, after which participants were to indicate their estimated duration of the star field by manipulating a slider using the left and right keys on the keyboard. During the first ten (training trials) trials, they were furthermore presented with an extra questionnaire item, in which participants indicated their sense of control on a visual-analogue scale of 0-100. Furthermore, the lab assistant explicitly monitored the experiment and informed the participant if their timing was too inaccurate (if the 8 s and 30 s trials showed negligible difference in estimated duration), so as to make sure participants performed the task correctly.
EEG processing
Calibration data were used to design classifier models, including training of common spatial pattern (CSP) filters, feature optimization, and data classification using a support vector machine (SVM) to enable online classification of EEG data for a neuroadaptive design.
Common spatial patterns Raw, FPz-referenced EEG were re-referenced to the common average reference, after which a spatial filter was applied creating 11 averaged channels, comprising prefrontal (FP1, FP2), frontal- left (F7, F3), frontocentral (Fz, FC1, FC2), frontal-right (F8, F4), central-left (C3, CP5), central (Cz, CP1, CP2), central-right (C4, CP6), parietal-left (P7, P3), centroparietal (Pz), parietal-right (P4, P8), and occipital (O1, Oz, O2) sites. These channels were then bandpass-filtered in 5 frequency-bands: delta (1–4 Hz), theta (4–7 Hz), alpha (7–12 Hz), beta (13–29 Hz), and gamma (30–40 Hz), along with one band comprising the full range, omni (2–40 Hz). The data were then segmented in 8 s epochs starting with an offset of 1 s from triggers signalling cues for imagining either running faster and faster (i.e. 1 s after seeing the running pictogram) or standing still (the standing pictogram). These 8 s epochs following either imagery types were then used to train 6 CSP filters, obtaining 11-dimensional spatial filters for optimally separating the two conditions using the respective bandpass-filtered data.
Classifier training Following training of CSP filters, the same processing pipeline was applied, except before bandpass filtering, the 6 distinct CSPs were applied. Then, the initial epoching was followed by a time-based epoching, re-segmenting the data at an interval of 0.25 s into 3 s subepochs (i.e. every sub-epoch overlapping 2.75 s with the preceding). A Hilbert transform was used to obtain envelopes describing the amplitude of the filtered data, which was averaged to obtain single features (66 features in total, or 11 channel groups x 6 frequency bands). From these, we applied a feature selection training to determine the 33% and 50% most diagnostic features using the minimum redundancy maximum relevance method. For each of these two sets of features, we applied machine learning using SVM (C-SVC type) with a radial basis function, and a cost of either 10 or 100. Each of these 4 models we then tested using cross-validation testing with 15 partitions and selecting the model providing the highest cross-validation accuracy for use during the experiment.
Neuroadaptivity The most accurate model from classifier training was used in a similar processing stream as described in the classifier trainer, except without the initial label-based epoching to provide blind prediction of labels for standing versus running, outputting a single probability of the one versus the other at a rate of 4 Hz over LAN. Furthermore, average channel activity was communicated to the stimulus PC for online rejection of artefact activity using a variable absolute cut-off threshold. Specifically, a label was scored as artefact (A) if its epoch had voltage above the threshold (ca. 50–100), and otherwise as running (R) if the probability was lower than the running probability cut-off (0.01–0.5, 0.5 by default), or as standing (S) if it was higher than the variable standing probability threshold (0.5–0.99, 0.5 by default). Adaptation occurred also at 4 Hz, with at each interval, the last 3 labels (R, S, and A) being taken into account (i.e. over the last 0.75 s) and a majority vote was used to determine the final score. If A was the majority, or no majority was received, the star field velocity was not changed. In adaptive conditions, if R was the majority, the velocity was increased by 1 level; while if S was the majority, the velocity was decreased by 1 level. In static conditions, the velocity was kept at the minimum level (1). Finally, classifications in neuroadaptive conditions were recorded and replayed during pseudoadaptive conditions, with respect to its relative duration so the visual presentation of each of these trials precisely matched a previous neuroadaptive one. In cases where no recorded neuroadaptive trial was yet available for replay (e.g. if pseudoadaptive trials preceded neuroadaptive trials during training trials), a recording from a previous participant was used instead.
Offline analysis Since the machine-learning approach used a complex combination of multiple features that were selected for maximum accuracy, it provided limited insight into the specific neural mechanisms enabling its efficacy. We therefore recorded EEG data during the experiment for offline analysis in MATLAB (and EEGLAB88) to investigate the relationship between oscillatory activity and subjective experience of time. Following removal of bad channels from the EEG and re-referencing the remaining to the common average, ICA was performed on further filtered (1–45 Hz), experimental data. On average, 8.9 components were identified as related to eye, movement, or other artefacts. The ICA solution and weights from the remaining 23.1 components were used on the non-filtered data, back-projecting the activity onto the scalp. Following, for each trial, EEG recorded during the entire star field’s duration, plus an initial 2 s before and 1 s after were decomposed in the time/frequency domain using Morlet-tapered wavelets with 3/0.5 cycles in the time/frequency-domain, resulting in 100 (frequencies from 2 to 45 Hz) × 200 (times, each over 1670 ms of data) datapoints. These were then natural log-transformed and averaged within the star field’s presentation times. For the analysis, we used the averaged activity in 5 frequency bands, delta (2–4 Hz), theta (4–7 Hz), alpha (7–12.5 Hz), beta (13–29 Hz), and gamma (30–40 Hz), and four areas, frontal (F3, Fz, F4), central (C3, Cz, C4), parietal (P3, Pz, P4), and occipital (O1, Oz, O2).
Design and analysis
The experiment used an orthogonal within-participant design with two imagination tasks (running vs standing) crossed with two adaptivity conditions (static vs adaptive), plus one additional ‘placebo’ condition. These five conditions were in turn crossed with the three durations (8 vs 18 vs 30 s), and repeated six times in random order for a total of 90 experimental trials. The 10 training trials used all 5 conditions in paired arrangement with two randomly selected durations. That is, the first two trials were of the same condition but different durations (randomly selected from the three experimental durations), the next two trials would be another condition, and so on. Items for reporting control followed every second trial in a pair and asked participants to indicate their experience during the last two trials (i.e. of the same condition). Inferential analysis involved two parts. First, the confirmatory analysis used repeated measures ANOVAs with objective duration (8 vs 18 vs 30 s), imagery (running vs standing) and adaptivity (static vs adaptive) as factors and subjective duration (s) and perception of time judgements (0–100) as measures. Furthermore, as specified in the pre-registration, a-priori comparisons were conducted using paired samples t-tests between neuroadaptive and pseudoadaptive conditions. Second, to explore the operation of the neuroadaptive device, we designed an exploratory analysis using a repeated measures ANOVAs with imagery, adaptivity, frequency (delta vs theta vs alpha vs beta vs gamma), and location (frontal vs central vs parietal vs occipital) as factors and average power. To follow up and detect whether pseudoadaptive were dissociable from neuroadaptive conditions, a similar analysis was carried out but with condition (neuroadaptive vs pseudoadaptive), frequency, and location as factors.
Experiment 2
Participants
Of the 29 volunteers participating in Experiment 2, 11 reported being female and 18 male, all were between 19 and 48 years of age (\(M = 29.62\), \(SD = 7.65\)). Participants received full instructions regarding the nature of the experiment. However, they were not informed that the trials always had the same duration. Furthermore, while they were informed that a ‘placebo-like control condition’ would be used in the experiment, they were kept blind to its precise nature or moment of presentation.
Stimuli and procedure
Experiment 2 used the same stimuli, apparatus, and procedure as reported in Experiment 1 apart from three critical changes. First, participants were no longer asked explicitly to imagine running faster and faster or standing still but were rather given a freedom to choose what to imagine and when. As illustrated in Fig. 7, trials started with a cue combining the running and standing pictograms into a single image, suggesting they freely altered within a trial between the two states. Second, star fields were presented for 5 min, following which three survey items were presented: a retrospective judgement on the elapsed time, a passage of time judgement, and a judgement on experienced control. The two latter items were the same as in Experiment 1, while the estimated duration item now showed a visual scale ranging from 0.5 to 40 min in intervals of 0.5 min. Third, due to the free-choice nature of the task, only three conditions were used in this study: the neuroadaptive condition, the static condition, and the pseudoadaptive condition. In the neuroadaptive condition, star field speed varied with imagery while in the static condition, they moved at a static, slow speed. During pseudoadaptive conditions, the star field speed varied in matched visual control to neuroadaptive conditions, but not based on online detection (see experiment 1).
Trial procedure of Experiment 2. Participants were asked to freely imagine either running or standing still or combination of both while stars appeared. Following mental imagery movement, they were asked to report their experiences.
EEG processing
The online EEG analysis and implementation of neuroadaptivity were as described in Experiment 1. For the offline EEG analysis, the time/frequency decomposition was adapted for the continuous trial structure, with data comprising 300s of experimental trial data. Accordingly, the wavelet analysis utilised 2000 timepoints x 100 frequencies, each still describing 1670 ms of EEG.
Design
The experiment used a within-participant design with three conditions (neuroadaptive vs static vs pseudoadaptive) presented once in random order. For the analysis, we used three independent one-way repeated measures ANOVAs with condition (neuroadaptive, static, pseudoadaptive) as factor and control, estimated duration, and passage of time judgement as dependent variables. For any significant effect, planned (two-sided) contrasts were conducted comparing neuroadaptive with pseudoadaptive conditions. To explore if effects of conditions on time perception were more due to visual effects, or due to the feeling of control, we conducted an ANCOVA-style analysis using two mixed multilevel models, predicting estimated duration (model 1) and passage of time (model 2) from experimental conditions and their resulting effects on optic flow and control. Participant was used as a random factor to cluster the three trials within participants. One fixed factor, condition, was used to differentiate the three types of feedback (neuroadaptive vs static vs pseudoadaptive). Also included in the models were self-reported control and averaged visual speed as covariates, and the interaction between condition and control.
Data Availability
Data will be provided upon requesting them from the corresponding author (mspape@um.edu.mo).
References
Block, R. A. & Gruber, R. P. Time perception, attention, and memory: A selective review. Acta Psychol. 149, 129–133 (2014).
Vogel, D. H. V. et al. Flow and structure of time experience-concept, empirical validation and implications for psychopathology. Phenomenol Cognit. Sci. 19, 235–258 (2020).
Brown, J. F. On time perception in visual movement fields. Psychol. Forsch. 14, 233–248 (1931).
Brown, S. W. Time, change, and motion: The effects of stimulus movement on temporal perception. Percept. Psychophys. 57, 105–116 (1995).
Zakay, D. & Block, R. A. The role of attention in time estimation processes. Adv. Psychol. 115, 143–164 (1996).
Pan, Y. & Luo, Q.-Y. Working memory modulates the perception of time. Psychon. Bull. Rev. 19, 46–51 (2012).
Aschersleben, G. Temporal control of movements in sensorimotor synchronization. Brain Cognit. 48, 66–79 (2002).
Keller, P. E. Joint action in music performance. Enacting intersubjectivity. A cognitive and social perspective on the study of interactions 205–221 (2008).
Loehr, J. D., Kourtis, D., Vesper, C., Sebanz, N. & Knoblich, G. Monitoring individual and joint action outcomes in duet music performance. J Cogn. Neurosci. 25, 1049–1061 (2013).
Maes, P.-J., Leman, M., Palmer, C. & Wanderley, M. Action-based effects on music perception. Front. Psychol. 4, 1008 (2014).
Roy, M. M., Christenfeld, N. J. & McKenzie, C. R. Underestimating the duration of future events: Memory incorrectly used or memory bias?. Psychol. Bull. 131, 738 (2005).
Van Rijn, H., Kononowicz, T. W., Meck, W. H., Ng, K. K. & Penney, T. B. Contingent negative variation and its relation to time estimation: A theoretical evaluation. Front. Integr. Neurosci. 5, 91 (2011).
Walter, W. G., Cooper, R., Aldridge, V. J., McCALLUM, W. C. & Winter, A. L. Contingent negative variation: An electric sign of sensori-motor association and expectancy in the human brain. Nature 203, 380–384. https://doi.org/10.1038/203380a0 (1964).
Lewis, A. The experience of time in mental disorder. Proc. R. Soc. Med. 25, 611–620 (1931).
Ratcliffe, M. Varieties of temporal experience in depression. Journal of Medicine and Philosophy 37, 114–138 (2012).
Strauss, E. W. Disorders of personal time in depressive states. South. Med. J. 40, 254–259. https://doi.org/10.1097/00007611-194703000-00011 (1947).
Vogel, D. H. V., Krämer, K., Schoofs, T., Kupke, C. & Vogeley, K. Disturbed Experience of time in depression-evidence from content analysis. Frontiers in Human Neuroscience12 (2018).
Mate, G. Scattered: How Attention Deficit Disorder Originates and What You Can do About It (Penguin, New York, 2000).
Ptacek, R. et al. Clinical implications of the perception of time in attention deficit hyperactivity disorder (ADHD): A review. Med. Sci. Monit. Int. Med. J. Exp. Clin. Res. 25, 3918 (2019).
Vogel, D. H. V. et al. Interrupted time experience in autism spectrum disorder: Empirical evidence from content analysis. J. Autism Developmental Disorders 49, 22–33 (2019).
Vogeley, K. & Kupke, C. Disturbances of time consciousness from a phenomenological and a neuroscientific perspective. Schizophr. Bull. 33, 157–165 (2007).
Gibbon, J. Scalar expectancy theory and Weber’s law in animal timing. Psychol. Rev. 84, 279 (1977).
Treisman, M. Temporal rhythms and cerebral rhythms. Ann. N. Y. Acad. Sci. 423, 542–565 (1984).
Treisman, M. Temporal discrimination and the indifference interval: Implications for a model of the “ internal clock’’. Psychol. Monogr. General Appl. 77, 1 (1963).
Gil, S. & Droit-Volet, S. Emotional time distortions: The fundamental role of arousal. Cognit. Emot. 26, 847–862 (2012).
O’Regan, J. K. & Noë, A. A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939 (2001).
Hommel, B., Müsseler, J., Aschersleben, G. & Prinz, W. The Theory of Event Coding (TEC): A framework for perception and action planning. The Behavioral and brain sciences 24, 849–878 (2001) (discussion 878–937).
Wittmann, M., Simmons, A. N., Aron, J. L. & Paulus, M. P. Accumulation of neural activity in the posterior insula encodes the passage of time. Neuropsychologia 48, 3110–3120 (2010).
Wittmann, M. Embodied Time: The Experience of Time, the Body, and the Self (MIT Press, Cambridge, 2014).
Münsterberg, H. Beiträge zur Experimentellen Psychologie Vol. 2 (Akademische Verlagsbuchhandlung von J. C. B. Mohr, Freiburg, 1889).
Kaski, D. et al. Temporoparietal encoding of space and time during vestibular-guided orientation. Brain 139, 392–403 (2016).
Yokosaka, T., Kuroki, S., Nishida, S. & Watanabe, J. Apparent time interval of visual stimuli is compressed during fast hand movement. PLoS ONE 10, e0124901 (2015).
Tomassini, A. & Morrone, M. C. Perceived visual time depends on motor preparation and direction of hand movements. Sci. Rep. 6, 27947. https://doi.org/10.1038/srep27947 (2016).
Yon, D., Edey, R., Ivry, R. B. & Press, C. Time on your hands: Perceived duration of sensory events is biased toward concurrent actions. J. Exp. Psychol. Gen. 146, 182 (2017).
De Kock, R., Zhou, W., Joiner, W. M. & Wiener, M. Slowing the body slows down time perception. Elife 10, e63607 (2021).
Spapé, M. M., Serrien, D. J. & Ravaja, N. 3-2-1, action! a combined motor control-temporal reproduction task shows intentions, motions, and consequences alter time perception. Heliyon9 (2023).
Wiener, M., Zhou, W., Bader, F. & Joiner, W. M. Movement improves the quality of temporal perception and decision-making. eneuro6 (2019).
Spapé, M. M., Harjunen, V. J. & Ravaja, N. Time to imagine moving: Simulated motor activity affects time perception. Psychonomic Bull. Rev. 29, 819–827 (2022).
Jeannerod, M. Neural simulation of action: A unifying mechanism for motor cognition. Neuroimage14, S103–S109 (2001). Publisher: Citeseer.
Decety, J., Jeannerod, M. & Prablanc, C. The timing of mentally represented actions. Behav. Brain Res. 34, 35–42 (1989).
Jeannerod, M. & Decety, J. Mental motor imagery: A window into the representational stages of action. Curr. Opin. Neurobiol. 5, 727–732 (1995).
Krol, L. R. & Zander, T. O. Passive BCI-based neuroadaptive systems. In GBCIC (2017).
Zakay, D. Time estimation methods-Do they influence prospective duration estimates?. Perception 22, 91–101 (1993).
Wearden, J. H. Passage of time judgements. Conscious. Cogn. 38, 165–171. https://doi.org/10.1016/j.concog.2015.06.005 (2015).
Jording, M., Vogel, D. H., Viswanathan, S. & Vogeley, K. Dissociating passage and duration of time experiences through the intensity of ongoing visual change. Sci. Rep. 12, 1–15 (2022).
Bindra, D. & Waksberg, H. Methods and terminology in studies of time estimation. Psychol. Bull. 53, 155 (1956).
Droit-Volet, S. & Dambrun, M. Awareness of the passage of time and self-consciousness: What do meditators report?. PsyCh J. 8, 51–65 (2019).
Formaggio, E., Storti, S. F., Cerini, R., Fiaschi, A. & Manganotti, P. Brain oscillatory activity during motor imagery in EEG-fMRI coregistration. Magn. Reson. Imaging 28, 1403–1412 (2010).
Bauer, M., Oostenveld, R., Peeters, M. & Fries, P. Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas. J. Neurosci. 26, 490–501 (2006).
Yuan, H., Perdoni, C. & He, B. Relationship between speed and EEG activity during imagined and executed hand movements. J. Neural Eng. 7, 026001 (2010).
Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 29, 169–195 (1999).
Toscani, M., Marzi, T., Righi, S., Viggiano, M. P. & Baldassi, S. Alpha waves: A neural signature of visual suppression. Exp. Brain Res. 207, 213–219 (2010).
Elsner, B. & Hommel, B. Effect anticipation and action control. J. Exp. Psychol. Hum. Percept. Perform. 27, 229 (2001).
Haggard, P. & Clark, S. Intentional action: Conscious experience and neural prediction. Conscious Cognit. 12, 695–707 (2003).
James, W. The Principles of Psychology (Dover Publications, New York, 1890).
Shin, Y. K., Proctor, R. W. & Capaldi, E. J. A review of contemporary ideomotor theory. Psychol. Bull. 136, 943 (2010).
Pfister, R., Kiesel, A. & Melcher, T. Adaptive control of ideomotor effect anticipations. Acta Physiol. (Oxf) 135, 316–322 (2010).
Pope, A. T., Bogart, E. H. & Bartolome, D. S. Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195. https://doi.org/10.1016/0301-0511(95)05116-3 (1995).
Pfurtscheller, G. & Neuper, C. Motor imagery and direct brain-computer communication. Proc. IEEE 89, 1123–1134. https://doi.org/10.1109/5.939829 (2001).
Gonzalez-Rosa, J. J. et al. Action observation and motor imagery in performance of complex movements: Evidence from EEG and kinematics analysis. Behav. Brain Res. 281, 290–300 (2015).
Rimbert, S., Lindig-León, C., Fedotenkova, M. & Bougrain, L. Modulation of beta power in eeg during discrete and continuous motor imageries. In: 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 333–336 (IEEE, 2017).
Spapé, M. et al. Brain-computer interface for generating personally attractive images. IEEE Trans. Affect. Comput. 14, 1–13 (2021).
Berger, H. Über das elektrenkephalogramm des menschen. Eur. Arch. Psychiatry Clin. Neurosci. 87, 527–570 (1929).
Benedek, M., Schickel, R. J., Jauk, E., Fink, A. & Neubauer, A. C. Alpha power increases in right parietal cortex reflects focused internal attention. Neuropsychologia 56, 393–400 (2014).
Serrien, D. J. & Spapé, M. M. The role of hand dominance and sensorimotor congruence in voluntary movement. Exp. Brain Res. 199, 195–200 (2009).
Serrien, D. J., Sovijärvi-Spapé, M. M. & Farnsworth, B. Bimanual control processes and the role of handedness. Neuropsychology 26, 802–807. https://doi.org/10.1037/a0030154 (2012).
Fairhall, S. L., Kirk, I. J. & Hamm, J. P. Volition and the idle cortex: Beta oscillatory activity preceding planned and spontaneous movement. Conscious. Cogn. 16, 221–228 (2007).
Tallon-Baudry, C. & Bertrand, O. Oscillatory gamma activity in humans and its role in object representation. Trends Cogn. Sci. 3, 151–162 (1999).
Fries, P., Nikolić, D. & Singer, W. The gamma cycle. Trends Neurosci. 30, 309–316 (2007).
Widmann, A., Gruber, T., Kujala, T., Tervaniemi, M. & Schröger, E. Binding symbols and sounds: Evidence from event-related oscillatory gamma-band activity. Cereb. Cortex 17, 2696–2702 (2007).
Gupta, D. S. & Chen, L. Brain oscillations in perception, timing and action. Curr. Opin. Behav. Sci. 8, 161–166 (2016).
Matthews, W. J. & Meck, W. H. Temporal cognition: Connecting subjective time to perception, attention, and memory. Psychol. Bull. 142, 865 (2016).
De Kock, R., Gladhill, K. A., Ali, M. N., Joiner, W. M. & Wiener, M. How movements shape the perception of time. Trends Cogn. Sci. 25, 950–963. https://doi.org/10.1016/j.tics.2021.08.002 (2021).
Todd, N. P. M., O’Boyle, D. J. & Lee, C. S. A sensory-motor theory of rhythm, time perception and beat induction. J. New Music Res. 28, 5–28 (1999).
Haggard, P., Clark, S. & Kalogeras, J. Voluntary action and conscious awareness. Nat. Neurosci. 5, 382–385 (2002).
Wenke, D. & Haggard, P. How voluntary actions modulate time perception. Exp. Brain Res. 196, 311–318 (2009).
Vogel, D. H., Jording, M., Esser, C., Weiss, P. H. & Vogeley, K. Temporal binding is enhanced in social contexts. Psychon. Bull. Rev. 28, 1545–1555 (2021).
Tse, P. U., Intriligator, J., Rivest, J. & Cavanagh, P. Attention and the subjective expansion of time. Percept. Psychophys. 66, 1171–1189 (2004).
Decety, J., Jeannerod, M., Germain, M. & Pastene, J. Vegetative response during imagined movement is proportional to mental effort. Behav. Brain Res. 42, 1–5 (1991).
Kunz, B. R., Creem-Regehr, S. H. & Thompson, W. B. Evidence for motor simulation in imagined locomotion. J. Exp. Psychol. Human Percept. Perform. 35, 1458 (2009).
Brown, S. W. & Boltz, M. G. Attentional processes in time perception: Effects of mental workload and event structure. J. Exp. Psychol. Hum. Percept. Perform. 28, 600 (2002).
Thönes, S., Arnau, S. & Wascher, E. Cognitions about time affect perception, behavior, and physiology: A review on effects of external clock-speed manipulations. Conscious. Cognit. 63, 99–109 (2018).
Rutrecht, H., Wittmann, M., Khoshnoud, S. & Igarzábal, F. A. Time speeds up during flow states: A study in virtual reality with the video game thumper. Timing Time Percept. 9, 353–376 (2021).
Gable, P. A. & Poole, B. D. Time flies when you’re having approach-motivated fun: Effects of motivational intensity on time perception. Psychol. Sci. 23, 879–886 (2012).
Harjunen, V. J., Spapé, M. & Ravaja, N. Anticipation of aversive visual stimuli lengthens perceived temporal duration. Psychol. Res. 86, 1230–1238 (2022).
Csikszentmihalyi, M. & Csikzentmihaly, M. Flow: The Psychology of Optimal Experience Vol. 1990 (Harper & Row, New York, 1990).
Renard, Y. et al. Openvibe: An open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence 19, 35–53 (2010).
Delorme, A. & Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009 (2004).
Acknowledgements
This research was funded by the EU; ERC Horizon 2020 VirtualTimes project.
Author information
Authors and Affiliations
Contributions
M.S. conceived the experiments, analysed the results, and wrote the first draft. I.A. programmed and piloted the experiments, and wrote the first draft of the method sections. V.H. helped with analysis and wrote part of the analysis. G.J. and N.R. contributed to conceptualisation and all authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Spapé, M., Ahmed, I., Harjunen, V. et al. A neuroadaptive interface shows intentional control alters the experience of time. Sci Rep 15, 9495 (2025). https://doi.org/10.1038/s41598-025-93204-0
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41598-025-93204-0
