Key Points
-
Afferent delays in the visual system are long and often lead to the mislocalization of objects. When our eyes or visible objects move, temporal errors in transmitting and processing neural signals can easily translate into spatial errors. This review considers three cases in which this occurs: when we pursue a moving target, when we try to localize a target that is presented just before a saccadic eye movement, and when we try to locate a moving stimulus with respect to a flashed one.
-
During smooth pursuit, an image of the moving target is always on the fovea. However, when that image reaches the level of perception, the eye is already pointing further along the anticipated target trajectory. Therefore, 'what-we-see' might not correspond to 'where-we-are-looking-at-that-instant'. Experimental data indicate that this is indeed the case. If so, our brain does not attempt to compensate for visual afferent delays.
-
The delay problem encountered in smooth pursuit is exacerbated during saccades because they are so fast. A large saccade can be performed during the time it takes for the image of a flashed object to reach the level of perception. How, then, can we localize the object? In the absence of any other visual cue, we have to rely on internal information about the trajectory and time course of the performed saccade. Experimental data show that this information (called the eye-position signal, or EPS) is available, although it is quite distorted. The distortion causes systematic mislocalizations, which affect stimuli presented not only during a saccade, but also before and after, when the eye is stable.
-
A number of illusions caused by stimulus motion are strikingly similar to perisaccadic mislocalization. The best known is the flash-lag effect, in which the perceived spatial relationship between a moving object and a stable one that is briefly flashed in the dark seems to be distorted. We argue that the cause of the illusion might be the indeterminacy of the exact time of an event (such as the flash). Recent data show that our current models of peaks of activity in visual maps are too simplistic to account for localization in the presence of change.
-
Analogous misperceptions can occur with other types of change, apart from stimulus–retinal slip. The problem can be formulated in general terms: when a variable changes (eye, body or stimulus variable), even predictably, how does the brain determine its state at the instant specified by an event? To answer this question, we have to understand how the brain encodes a changing variable (for example, position versus velocity signals), and how it reads that code (continuously or by sampling).
Abstract
Reviews on the visual system generally praise its amazing performance. Here we deal with its biggest weakness: sluggishness. Inherent delays lead to mislocalization when things move or, more generally, when things change. Errors in time translate into spatial errors when we pursue a moving object, when we try to localize a target that appears just before a gaze shift, or when we compare the position of a flashed target with the instantaneous position of a continuously moving one (or one that appears to be moving even though no change occurs in the retinal image). Studying such diverse errors might rekindle our thinking about how the brain copes with real-time changes in the world.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout






Similar content being viewed by others
References
Hazelhoff, F. & Wiersma, H. Die Wahrnehmungszeit. Z. Psychol. 96, 181–188 (1924).
Schmolesky, M. T. et al. Signal timing across the macaque visual system. J. Neurophysiol. 79, 3272–3278 (1998).
Ward, F. in Eye Movements and Psychological Processes (eds Monty, R. A. & Senders, J. W.) 289–297 (Erlbaum, Hillsdale, New Jersey, 1976).
Mateeff, S., Yakimoff, N. & Dimitrov, G. Localization of brief stimuli during pursuit eye movements. Acta Psychol. (Amst.) 48, 133–140 (1981).
Brenner E., Smeets, J. B. J. & van den Berg, A. V. Smooth eye movements and spatial localisation. Vision Res. 41, 2253–2259 (2001).
Smeets, J. B. J. & Brenner, E. Perception and action are based on the same visual information: distinction between position and velocity. J. Exp. Psychol. Hum. Percept. Perform. 21, 19–31 (1995).
Newsome, W. T., Wurtz, R. H. & Komatsu, H. Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. J. Neurophysiol. 60, 604–620 (1988).
Paillard, J. Quelques données psychophysiologiques relatives au déclenchement de la commande motrice. Année Psychol. 48, 28–47 (1948).
Aschersleben, G. & Prinz, W. Synchronizing actions with events: the role of sensory information. Percept. Psychophys. 57, 305–317 (1995).
Aschersleben, G. in Cognitive Contributions to the Perception of Spatial and Temporal Events (eds Aschersleben, G., Bachmann, T. & Müsseler, J.) 293–318 (Elsevier, Amsterdam, 1999).
Churchland, M. M. & Lisberger, S. G. Experimental and computational analysis of monkey smooth pursuit eye movements. J. Neurophysiol. 86, 741–759 (2001).
Dassonville, P., Schlag, J. & Schlag-Rey, M. Oculomotor localization relies on a damped representation of saccadic eye displacement in human and nonhuman primates. Vis. Neurosci. 9, 261–269 (1992).
Dassonville, P., Schlag, J. & Schlag-Rey, M. Direction constancy in the oculo-motor system. Curr. Dir. Psychol. Sci. 2, 143–147 (1993).
Mitrani, L., Mateeff, S. & Yakimoff, N. Smearing of the retinal image during voluntary saccadic eye movements. Vision Res. 10, 405–409 (1970).
Matin, L. & Pearce, D. G. Visual perception of direction for stimuli flashed during voluntary saccadic eye movement. Science 148, 1485–1488 (1965).Although not the first one on this topic, this is the most influential paper to have drawn attention to mislocalization near the time of saccades.
Bischof, F. & Kramer, E. Untersuchungen und Überlegungen zur Richtungswahrnehmung bei willkürlichen sakkadischen Augenbewegungen. Psychol. Forsch. 32, 185–218 (1968).
Hallett, P. E. & Lightstone, A. D. Saccadic eye movements toward stimuli triggered by prior saccades. Vision Res. 16, 99–106 (1976).This study provided evidence that the brain uses an EPS to locate visual targets in the dark.
Hansen, R. M. & Skavenski, A. A. Accuracy of spatial localization near the time of saccadic eye movements. Vision Res. 25, 1077–1082 (1985).
Honda, H. Spatial localization in saccade and pursuit-eye-movement conditions: a comparison of perceptual and motor measures. Percept. Psychophys. 38, 41–46 (1985).
Honda, H. in Attention and Performance XIII: Motor Representation and Control. (ed. Jeannerod, M.) 567–582 (Erlbaum, Hillsdale, New Jersey, 1990).
Dassonville, P., Schlag, J. & Schlag-Rey, M. The use of egocentric and exocentric location cues in saccadic programming. Vision Res. 35, 2191–2199 (1995).
Schlag, J. & Schlag-Rey, M. Illusory localization of stimuli flashed in the dark before saccades. Vision Res. 35, 2347–2357 (1995).
Bockisch, C. J. & Miller, J. M. Different motor systems use similar damped extraretinal eye position information. Vision Res. 39, 1025–1038 (1999).
Matin, L. et al. Oculoparalytic illusion: visual-field dependent spatial mislocalizations by humans partially paralyzed with curare. Science 216, 198–201 (1982).Probably the most elegant demonstration of the role of the internal EPS in stimulus localization and of its overriding by ambient visual cues.
Bridgeman, B. & Delgado, D. Sensory effects of eye press are due to efference. Percept. Psychophys. 36, 482–484 (1984).
Grüsser, O.-J., Krizic, A. & Weiss, L.-R. After-image movement during saccades in dark. Vision Res. 27, 215–226 (1987).
Mergner, T., Nasios, G., Maurer, C. & Becker, W. Visual localization in space: interaction of retinal, eye position, vestibular and neck proprioceptive information. Exp. Brain Res. 141, 33–51 (2001).
Goldberg, M. E. & Bruce, C. J. Primate frontal eye fields. III. Maintenance of spatially accurate saccade signal. J. Neurophysiol. 64, 489–508 (1990).
Moschovakis, A. K., Karalelas, A. B. & Highstein, S. N. Structure–function relationships in the primate superior colliculus. II. Morphological identity of presaccadic neurons. J. Neurophysiol. 60, 263–302 (1988).
Quaia, C., Optican, L. M. & Goldberg, M. E. The maintenance of spatial accuracy by the perisaccadic remapping of visual receptive fields. Neural Netw. 11, 1229–1240 (1998).
Zipser, D. & Andersen, R. A. A back-propagation programmed network that stimulates response properties of a subset of posterior parietal neurons. Nature 331, 679–684 (1988).
Duhamel, J.-R., Colby, C. L. & Goldberg, M. E. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92 (1992).
Umeno, M. M. & Goldberg, M. E. Spatial processing in the monkey's frontal eye field. Predictive visual responses. J. Neurophysiol. 78, 1373–1383 (1997).
Walker, M. F., FitzGibbon, E. J. & Goldberg, M. E. Neurons in the monkey superior colliculus predict the result of impending saccadic eye movements. J. Neurophysiol. 73, 1988–2003 (1995).
Honda, H. Perceptual localization of visual stimuli flashed during saccades. Percept. Psychophys. 45, 162–174 (1989).
Boucher, L., Groh, J. M. & Hughes, H. C. Afferent delays and the mislocalization of perisaccadic stimuli. Vision Res. 41, 2631–2644 (2001).
Pouget, A., Albright, T. & Sejnowski, T. A dynamic model for computing the position of an object from its retinal location and eye position. Soc. Neurosci. Abstr. 18, 1396 (1992).
Pola, J. R & Wyatt, H. J. The time course of the extraretinal signal for saccade-contingent perceived direction may be as fast as the saccade. Soc. Neurosci. Abstr. 31, 58.19 (2001).
Sperling, G. in Eye Movements and their Role in Visual and Cognitive Processes (ed. Kowler, E.) 307–351 (Elsevier, Amsterdam, 1990).
Volkmannn, F. C. & Moore, R. K. in Visual Psychophysics and Physiology (eds Armington, J. C., Krauskopf, J. & Wooden, B. R.) 353–362 (Academic, New York, 1978).
Dassonville, P. Haptic localization and internal representation of the hand in space. Exp. Brain Res. 106, 434–448 (1995).
Hershberger, W. Saccadic eye movements and the perception of visual direction. Percept. Psychophys. 41, 35–44 (1987).
Kubischik, M. & Bremmer, F. Peri-saccadic space representation in monkey inferior parietal cortex. Soc. Neurosci. Abstr. 25, 1164 (1999).
Mays, L. E. & Sparks, D. L. Dissociation of visual and saccade-related responses in superior colliculus neurons. J. Neurophysiol. 43, 207–232 (1980).
Robinson, D. A. & Fuchs, A. F. Eye movements evoked by stimulation of frontal eye fields. J. Neurophysiol. 32, 637–648 (1969).
Sparks, D. L. & Mays, L. E. Spatial localization of saccade targets. I. Compensation for stimulus induced perturbations in eye position. J. Neurophysiol. 49, 45–74 (1983).
Schlag, J. & Schlag-Rey, M. Does microstimulation evoke fixed vector saccades by generating their vector or by specifying their goal? Exp. Brain Res. 68, 442–444 (1987).
Schlag, J. & Schlag-Rey, M. Colliding saccades may reveal the secret of their marching orders. Trends Neurosci. 13, 410–415 (1990).
Dassonville, P., Schlag, J. & Schlag-Rey, M. The frontal eye field provides the goal of saccadic eye movement. Exp. Brain Res. 89, 300–310 (1992).
Schlag, J., Schlag-Rey, M. & Dassonville, P. Interactions between natural and electrically evoked saccades. II. At what time is eye position sampled as a reference for the localization of a target? Exp. Brain Res. 76, 548–558 (1989).
Schlag-Rey, M., Schlag, J. & Dassonville, P. Interactions between natural and electrically evoked saccades. I. Differences between sites carrying retinal error and motor command signals in monkey superior colliculus. Exp. Brain Res. 76, 537–547 (1989).
Dominey, P. F., Schlag, J., Schlag-Rey, M. & Arbib, M. A. Colliding saccades evoked by frontal eye field stimulation: artifact or evidence for an oculomotor compensatory mechanism underlying double-step saccades. Biol. Cybern. 76, 41–52 (1997).
Brenner, E. & Cornelissen, F. W. Separate simultaneous processing of egocentric and relative positions. Vision Res. 40, 2557–2564 (2000).
Honda, H. Saccade-contingent displacement of the apparent position of visual stimuli flashed on a dimly illuminated structured background. Vision Res. 33, 709–716 (1993).
Karn, K. S., Moller, P. & Hayhoe, M. M. Reference frames in saccadic targeting. Exp. Brain Res. 115, 267–282 (1997).
Hahnloser, R., Douglas, R. G., Mahowald, M. & Hepp, K. Feedback interactions between neuronal pointers and maps for attentional processing. Nature Neurosci. 2, 746–752 (1999).
Cai, R. H., Pouget, A., Schlag-Rey, M. & Schlag, J. Perceived geometrical relationships affected by eye-movement signals. Nature 386, 601–604 (1997).
Ross, J., Morrone, M. C. & Burr, D. C. Compression of visual space before saccades. Nature 386, 598–601 (1997).
Morrone, M. C., Ross, J. & Burr, D. C. Apparent position of visual targets during real and simulated saccadic eye movements. J. Neurosci. 17, 7941–7953 (1997).
Lappe, M., Awater, H. & Krekelberg, B. Postsaccadic visual references generate presaccadic compression of space. Nature 403, 892–895 (2000).
Ross, J., Morrone, M. C., Goldberg, M. E. & Burr, D. C. Changes in visual perception at the time of saccades. Trends Neurosci. 24, 113–121 (2001).
van Beers, R. J., Wolpert, D. M. & Haggard, P. Sensorimotor integration compensates for visual localization errors during smooth pursuit eye movements. J. Neurophysiol. 85, 1914–1922 (2001).
Deubel, H., Schneider, W. X. & Bridgeman, B. Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Res. 36, 985–996 (1996).
Fröhlich, F. W. Über die Messung der Empfindungszeit. Z. Sinnesphysiol. 54, 57–78 (1923).
Metzger, W. Versuch einer gemeinsamen Theorie der Phänomene Fröhlichs und Hazelhoffs und Kritik ihrer Vefahren zur Messung der Empfindungszeit. Psychol. Forsch. 16, 176–200 (1931).
MacKay, D. M. Perceptual stability of a stroboscopically lit visual field containing self-luminous objects. Nature 181, 507–508 (1958).
Nijhawan, R. Motion extrapolation in catching. Nature 370, 256–257 (1994).This short paper renewed interest in the flash-lag phenomenon and started an intense debate.
Nijhawan, R. The flash-lag phenomenon: object motion and eye movements. Perception 30, 263–282 (2001).
Purushothaman, G., Patel, S. S., Bedell, H. E. & Ogmen, H. Moving ahead through differential visual latency. Nature 396, 424 (1998).
Whitney, D. & Murakami, I. Latency difference, not spatial extrapolation. Nature Neurosci. 1, 656–657 (1998).
Whitney, D., Murakami, I. & Cavanagh, P. Illusory spatial offset of a flash relative to a moving stimulus is caused by differential latencies for moving and flashed stimuli. Vision Res. 40, 137–149 (2000).
Murakami, I. A flash-lag effect in random motion. Vision Res. 41, 3101–3119 (2001).
Baldo, M. V. C. & Klein, S. A. Extrapolation or attention shift? Nature 378, 565–566 (1995).
Super, H., Spekreijse, H. & Lamme, V. A. F. Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nature Neurosci. 4, 304–310 (2001).
Allik, J. & Kreegipuu, K. Multiple visual latency. Psychol. Sci. 9, 135–138 (1998).
Arnold, D. H., Clifford, C. W. G. & Wenderoth, P. Asynchronous processing in vision: color leads motion. Curr. Biol. 11, 596–600 (2001).
Bartels, A. & Zeki, S. The theory of multistage integration in the visual brain. Proc. R. Soc. Lond. B 265, 2327–2332 (1998).
Berry, M. J., Brivanlou, I. H., Jordan, T. A. & Meister, M. Anticipation of moving stimuli by the retina. Nature 398, 334–338 (1999).
Müller, K., Aschersleben, G., Koch, R., Freund, H. & Prinz, W. in Cognitive Contributions to the Perception of Spatial and Temporal Events (eds Aschersleben, G., Bachmann, T. & Müsseler, J.) 233–250 (Elsevier, Amsterdam, 1999)
Lappe, M. & Krekelberg, B. The position of moving objects. Perception 27, 1437–1449 (1998).
Krekelberg, B. & Lappe, M. Temporal recruitment along the trajectory of moving objects and the perception of position. Vision Res. 39, 2669–2679 (1999).
Watanabe, K., Nijhawan, R., Khurana, B. & Shimojo, S. Perceptual organization of moving stimuli modulates the flash-lag effect. J. Exp. Psychol. Hum. Percept. Perform. 27, 879–894 (2001).
Brenner, E. & Smeets, J. B. J. Motion extrapolation is not responsible for the flash-lag effect. Vision Res. 40, 1645–1648 (2000).
Eagleman, D. M. & Sejnowski, T. J. Motion integration and postdiction in visual awareness. Science 287, 2036–2038 (2000).
Cai, R. H. & Schlag, J. Asynchronous feature binding and the flash-lag illusion. Invest. Ophthalmol. Vis. Sci. 42, 3830 (2001).
Cai, R. H., Schlag-Rey, M. & Schlag, J. Displacement of the moving bar exists in the flash-lag effect. Soc. Neurosci. Abstr. 26, 1502 (2000).
Snowden, R. J. Shifts in perception following adaptation to visual motion. Curr. Biol. 8, 1343–1345 (1998).
Schlag, J., Cai, R. H., Dorfman, A., Mohempour, A. & Schlag-Rey, M. Extrapolating movement without retinal motion. Nature 403, 38–39 (2000).
Cai, R. H., Jacobson, K., Baloh, R., Schlag-Rey, M. & Schlag, J. Vestibular signals can distort the perceived spatial relationship of retinal stimuli. Exp. Brain Res. 135, 275–278 (2000).
Bachman, T. & Poder, E. Change in feature space is not necessary for the flash-lag effect. Vision Res. 41, 1103–1106 (2001).
Sheth, B., Nijhawan, R. & Shimojo, S. Changing objects lead briefly flashed ones. Nature Neurosci. 3, 489–495 (2000).
Fu, Y.-X., Shen, Y. & Dan, Y. Motion-induced perceptual extrapolation of blurred visual targets. J. Neurosci. 21, RC172 (2001).
Deubel, H., Irwin, D. E. & Schneider, W. X. in Current Oculomotor Research: Physiological and Psychological Aspects (eds Becker, W., Deubel, H. & Mergner, T.) 65–70 (Plenum, New York, 1999).
Krekelberg, B. The persistence of position. Vision Res. 41, 529–539 (2001).
Krekelberg, B. & Lappe, M. Neuronal latencies and the position of moving objects. Trends Neurosci. 24, 335–339 (2001).
Tolias, A. S. et al. Eye movements modulate visual receptive fields of V4 neurons. Neuron 29, 757–767 (2001).
Paillard, J. & Amblard, B. in Brain Mechanisms and Spatial Vision (eds Ingle, D. J., Jeannerod, M. & Lee, D. N.) 299–329 (Martinus Nijhoof, Dordrecht, 1985).
Priebe, N. J., Churchland, M. M. & Lisberger, S. G. Reconstruction of target speed for the guidance of pursuit eye movements. J. Neurosci. 21, 3196–3206 (2001).
Krekelberg, B. & Lappe, M. A model of the perceived relative positions of moving objects based upon a slow averaging process. Vision Res. 40, 201–215 (2000).
Rao, R. P. N., Eagleman, D. M. & Sejnowski, T. J. Optimal smoothing in visual motion perception. Neural Comput. 13, 1243–1253 (2001).
Van der Heiden, A. H. C., Müsseler, J. & Bridgeman, B. in Cognitve Contributions to the Perception of Spatial and Temporal Events (eds Aschersleben, G., Bachmann, T. & Müsseler, J.) 19–37 (Elsevier, Amsterdam, 1999).
De Valois, R. L. & De Valois, K. K. Vernier acuity with stationary moving Gabors. Vision Res. 31, 1619–1626 (1991).
Nishida, S. & Johnston, A. Influence of motion signals on the perceived position of spatial pattern. Nature 397, 610–612 (1999).
Whitney, D. & Cavanagh, P. Motion distorts visual space: shifting the perceived position of remote stationary objects. Nature Neurosci. 3, 954–959 (2000).
Tian, J.-R. & Lynch, J. C. Corticocortical input to the smooth and saccadic eye movement subregions of the frontal eye field in Cebus monkeys. J. Neurophysiol. 76, 2754–2771 (1996).
Andersen, R. A., Asanuma, C., Essick, G. & Siegel, R. M. Corticocortical connections of anatomically and physiologically defined subdivisions within the inferior parietal lobule. J. Comp. Neurol. 296, 65–113 (1990).
Boussaoud, D., Ungerleider, L. G. & Desimone, R. Pathways for motion analysis: cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque. J. Comp. Neurol. 296, 462–495 (1990).
Cavada, C. & Goldman-Rakic, P. S. Posterior parietal cortex in rhesus monkey. I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J. Comp. Neurol. 287, 393–421 (1989).
Huerta, M. F. & Kaas, J. H. Supplementary eye field as defined by intracortical microstimulation: connections in macaques. J. Comp. Neurol. 293, 299–330 (1990).
Schall, J. D., Morel, A., King, D. J. & Bullier, J. Topography of visual cortex connections with frontal eye field in macaque: convergence and segregation of processing streams. J. Neurosci. 15, 4464–4487 (1995).
Yin, J., Schlag-Rey, M. & Schlag, J. Comparison of origins of projections from LIP to SEF and FEF in primate. Soc. Neurosci Abstr. 20, 145 (1994).
Schlag-Rey, M. & Schlag, J. in The Neurobiology of Saccadic Eye Movements (eds Wurtz, R. H. & Goldberg, M. E.) 361–390 (Elsevier, Amsterdam, 1989)
Kraulzlis, R. J. & Stone, L. S. Tracking with the mind's eye. Trends Neurosci. 22, 544–550 (1999).
Acknowledgements
Support was provided by grants from the National Institutes of Health. We thank E. Brenner, P. Dassonville, R. Cai and J. Park for helpful comments.
Author information
Authors and Affiliations
Related links
Related links
FURTHER INFORMATION
MIT Encyclopedia of Cognitive Sciences
eye movements and visual attention
LINKS
Glossary
- CLOSED LOOP
-
A system in which the input is made dependent on the output by feedback.
- EFFERENCE COPY
-
A copy of a motor command that is sent back to the central nervous system to inform it of the executed movement.
- SPATIAL CONSTANCY
-
The perceptual assumption that objects are still where they were in the world when the retinal shift of their image is caused by our own movements.
Rights and permissions
About this article
Cite this article
Schlag, J., Schlag-Rey, M. Through the eye, slowly; Delays and localization errors in the visual system. Nat Rev Neurosci 3, 191 (2002). https://doi.org/10.1038/nrn750
Issue date:
DOI: https://doi.org/10.1038/nrn750
This article is cited by
-
Rat superior colliculus encodes the transition between static and dynamic vision modes
Nature Communications (2024)
-
Theoretical prediction of broadband ambient light optogenetic vision restoration with ChRmine and its mutants
Scientific Reports (2024)
-
The effect of impaired velocity signals on goal-directed eye and hand movements
Scientific Reports (2023)
-
Motion-in-depth effects on interceptive timing errors in an immersive environment
Scientific Reports (2021)
-
Heading representations in primates are compressed by saccades
Nature Communications (2017)