Abstract
In our environment, stimuli from different sensory modalities are initially processed within a temporal window of multisensory integration spanning several hundred milliseconds. During this window, stimulus processing is influenced not only by preceding and current information, but also by input that follows the stimulus. The computational mechanisms underlying crossmodal backward processing, which we refer to as crossmodal postdiction, are not well understood. We examined crossmodal postdiction in the Illusory Audiovisual (AV) Rabbit and Invisible AV Rabbit Illusions, in which postdiction occurs when flash-beep pairs are presented shortly before and shortly after a single flash or a single beep. We collected behavioral data from 32 participants and fitted four competing models: Bayesian Causal Inference (BCI), forced-fusion, forced-segregation, and non-postdictive BCI. The BCI model fit the data well and outperformed all other models. Building on previous findings that demonstrate causal inference during non-postdictive multisensory integration, our results show that the BCI framework can also explain crossmodal postdiction phenomena. Our findings suggest that the brain performs causal inference not only across concurrent sensory inputs but also across temporal windows, integrating information from past, present, and subsequent events across modalities to construct a unified percept.
Data availability
The collected data and generated datasets of this study are available upon reasonable request.
References
Sperling, G. The information available in brief visual presentations. Psychol. Monogr. Gen. Appl. 74(11), 1–29 (1960).
Stiles, N. R. B., Armand, R. T., Tanguay, J. & Shimojo, S. Crossmodal postdiction: Conscious perception as revisionist history. J. Percept. Imaging 5, jpi0150 (2021).
Shimojo, S. Postdiction: Its implications on visual awareness, hindsight, and sense of agency. Front. Psychol. 5, 59746 (2014).
Liesner, M., Kirsch, W. & Kunde, W. The interplay of predictive and postdictive components of experienced selfhood. Conscious Cogn. 77, 102850 (2020).
Goldreich, D. & Tong, J. Prediction, postdiction, and perceptual length contraction: A Bayesian low-speed prior captures the cutaneous rabbit and related illusions. Front. Psychol. 4, 50645 (2013).
Noppeney, U. Perceptual inference, learning, and attention in a multisensory world. Annu. Rev. Neurosci. 44, 449–473 (2021).
Senkowski, D. & Engel, A. K. Multi-timescale neural dynamics for multisensory integration. Nat. Rev. Neurosci. 25(9), 625–642 (2024).
Colonius, H. & Diederich, A. Measuring multisensory integration: from reaction times to spike counts. Sci. Rep. 7(1), 1–11 (2017).
Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429–433 (2002).
Körding, K. P. et al. Causal inference in multisensory perception. PLoS ONE 2(9), e943 (2007).
Shams, L. & Beierholm, U. R. Causal inference in perception. Trends Cogn. Sci. 14(9), 425–432 (2010).
Shams, L. & Beierholm, U. Bayesian causal inference: A unifying neuroscience theory. Neurosci. Biobehav. Rev. 137, 104619 (2022).
Chandrasekaran, C. Computational principles and models of multisensory integration. Curr. Opin. Neurobiol. 43, 25–34 (2017).
Rohe, T., Ehlis, A. C. & Noppeney, U. The neural dynamics of hierarchical Bayesian causal inference in multisensory perception. Nat. Commun. 10(1), 1907 (2019).
Rohe, T. & Noppeney, U. Cortical hierarchies perform Bayesian causal inference in multisensory perception. PLoS Biol. 13(2), e1002073 (2015).
Shams, L., Ma, W. J. & Beierholm, U. Sound-induced flash illusion as an optimal percept. NeuroReport 16(17), 1923–1927 (2005).
Shams, L., Kamitani, Y. & Shimojo, S. What you see is what you hear. Nature 408(6814), 788 (2000).
Stiles, N. R. B., Li, M., Levitan, C. A., Kamitani, Y. & Shimojo, S. What you saw is what you will hear: Two new illusions with audiovisual postdictive effects. PLoS ONE 13(10), e0204217 (2018).
Alais, D. & Burr, D. The, “flash-lag” effect occurs in audition and cross-modally. Curr. Biol. 13(1), 59–63 (2003).
Colonius, H. & Diederich, A. The optimal time window of visual–auditory integration: A reaction time analysis. Front. Integr. Neurosci. 4, 1316 (2010).
van Wassenhove, V., Grant, K. W. & Poeppel, D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45(3), 598–607 (2007).
Andersen, T. S., Tiippana, K. & Sams, M. Factors influencing audiovisual fission and fusion illusions. Cogn. Brain Res. 21(3), 301–308 (2004).
Kok, P., Brouwer, G. J., van Gerven, M. A. J. & de Lange, F. P. Prior expectations bias sensory representations in visual cortex. J. Neurosci. 33(41), 16275–16284 (2013).
Colonius, H. & Diederich, A. Focused attention vs. crossmodal signals paradigm: Deriving predictions from the time-window-of-integration model. Front. Integr. Neurosci. 6, 62 (2012).
Stevenson, R. A. & Wallace, M. T. Multisensory temporal integration: Task and stimulus dependencies. Exp. Brain Res. 227(2), 249–261 (2013).
Tanguay, A. R., Stiles, N. R. B., Ganguly, I. & Shimojo, S. Time dependence of predictive and postdictive auditory-visual processing: The temporally extended audiovisual rabbit illusion. J. Vis. 19(10) (2019).
Vroomen, J. & Keetels, M. Perception of intersensory synchrony: A tutorial review. Atten. Percept. Psychophys. 72(4), 871–884 (2010).
Diederich, A. & Colonius, H. Crossmodal interaction in speeded responses: time window of integration model. Prog. Brain Res. 174, 119–135 (2009).
Krisst, L. C., Chan, A. Y. C., Stiles, N. R. B., Levitan, C. A. & Shimojo, S. Exploring the neural correlates of crossmodal illusory perception. J. Vis. 25(9), 2912 (2025).
Cederblad, M., Tanguay, A. R. Jr., Shimojo, S. & Stiles, N. R. B. The audiovisual rabbit illusion with illusory contours. J. Vis. 23(9), 5590 (2023).
Cao, Y., Summerfield, C., Park, H., Giordano, B. L. & Kayser, C. Causal inference in the multisensory brain. Neuron 102(5), 1076-1087.e8 (2019).
Aller, M. & Noppeney, U. To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference. PLoS Biol. 17(4), e3000210 (2019).
Khoei, M. A., Masson, G. S. & Perrinet, L. U. The flash-lag effect as a motion-based predictive shift. PLoS Comput. Biol. 13(1), e1005068 (2017).
Balz, J. et al. GABA concentration in superior temporal sulcus predicts gamma power and perception in the sound-induced flash illusion. Neuroimage 125, 724–730 (2016).
Bhattacharya, J., Shams, L. & Shimojo, S. Sound-induced illusory flash perception: Role of gamma band responses. NeuroReport 13(14), 1727–1730 (2002).
Rohe, T. & Noppeney, U. Distinct computational principles govern multisensory integration in primary sensory and association cortices. Curr. Biol. 26(4), 509–514 (2016).
Herzog, M. H., Drissi-Daoudi, L. & Doerig, A. All in good time: Long-lasting postdictive effects reveal discrete perception. Trends Cogn. Sci. 24(10), 826–837 (2020).
Mihalik, A., Noppeney, U. Causal inference in audiovisual perception. J. Neurosci. 40(34) (2020).
Bastos, A. M. et al. Canonical microcircuits for predictive coding. Neuron 76(4), 695–711 (2012).
Bastiaansen, M., Berberyan, H., Stekelenburg, J. J., Schoffelen, J. M. & Vroomen, J. Are alpha oscillations instrumental in multisensory synchrony perception?. Brain Res. 1734, 146744 (2020).
London, R. E. et al. EEG alpha power predicts the temporal sensitivity of multisensory perception. Eur. J. Neurosci. 55(11–12), 3241–3255 (2022).
Leonardelli, E. et al. Prestimulus oscillatory alpha power and connectivity patterns predispose perceptual integration of an audio and a tactile stimulus. Hum. Brain Mapp. 36(9), 3486–3498 (2015).
Grabot, L., Kayser, C. & van Wassenhove, V. Postdiction: When temporal regularity drives space perception through prestimulus alpha oscillations. eNeuro. 8(5) (2021).
Kleiner, M. et al. What’s new in psychtoolbox-3. Perception 36(14), 1–16 (2007).
The MathWorks Inc. MATLAB (The MathWorks, Inc., Natick, MA, 2024).
Samad, M., Chung, A. J. & Shams, L. Perception of body ownership is driven by Bayesian sensory inference. PLoS ONE 10(2), e0117178 (2015).
Wozny, D. R., Beierholm, U. R. & Shams, L. Probability matching as a computational strategy used in perception. PLoS Comput. Biol. 6(8), e1000871 (2010).
Acerbi, L., Ma, W. J. Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Adv Neural Inf Process Syst. 30 (2017).
Rohe, T., Hesse, K., Ehlis, A. C. & Noppeney, U. Multisensory perceptual and causal inference is largely preserved in medicated post-acute individuals with schizophrenia. PLoS Biol. 22(9), e3002790 (2024).
Raftery, A. E. Bayesian model selection in social research. Sociol. Methodol. 25, 111 (1995).
Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies - revisited. Neuroimage 84, 971–985 (2014).
Friston, K. J. Functional and effective connectivity in neuroimaging: A synthesis. Hum. Brain Mapp. 2(1–2), 56–78 (1994).
Acknowledgements
This research was supported by grants from the Deutsche Forschungsgemeinschaft (DFG) to TR (RO 5587/5 − 1) and DS (SE1859/10 − 1).
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
D.S., T.R., G.G., and J.M. contributed to the conception and design of the study. G.G. was responsible for data collection. T.R., G.G., and D.S. performed the data analysis and interpretation. D.S. led the drafting of the manuscript. All authors contributed to manuscript revision, read, and approved the final version. D.S. and T.R. supervised the project.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Günaydın, G., Moran, J.K., Rohe, T. et al. Causal inference shapes crossmodal postdiction in multisensory integration. Sci Rep (2026). https://doi.org/10.1038/s41598-026-36884-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-36884-6
