Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Revealing neurocognitive and behavioral patterns through unsupervised manifold learning of dynamic brain data

Nature Computational Science volume 5, pages 1238–1252 (2025) Cite this article

Subjects

Abstract

Dynamic brain data are becoming increasingly accessible, providing a gateway to understanding the inner workings of the brain in living participants. However, the size and complexity of the data pose a challenge in extracting meaningful information across various data sources. Here we introduce a generalizable unsupervised deep manifold learning for exploration of neurocognitive and behavioral patterns. Unlike existing methods that extract patterns directly from the input data, the proposed brain-dynamic convolutional-network-based embedding (BCNE) captures brain-state trajectories by analyzing temporospatial correlations within the data and applying manifold learning. The results demonstrate that BCNE effectively delineates scene transitions, underscores the involvement of different brain regions in memory and narrative processing, distinguishes dynamic learning processes and identifies differences between active and passive behaviors. BCNE provides an effective tool for exploring general neuroscience inquiries or individual-specific patterns.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematic of the brain-dynamic trajectory visualization pipeline.
The alternative text for this image may have been generated using AI.
Fig. 2: Visualization and analysis of the Sherlock fMRI dataset.
The alternative text for this image may have been generated using AI.
Fig. 3: Evaluation of the BCNE framework and behaviorally rated event analysis of the Sherlock fMRI dataset.
The alternative text for this image may have been generated using AI.
Fig. 4: Results of the rat hippocampus dataset.
The alternative text for this image may have been generated using AI.
Fig. 5: Results of the macaque dataset.
The alternative text for this image may have been generated using AI.

Similar content being viewed by others

Data availability

Source data are provided with this paper. This study utilized three publicly available datasets: the Sherlock fMRI dataset34 (preprocessing pipeline available via GitHub at https://github.com/KrishnaswamyLab/TPHATE), the hippocampus dataset35 and the macaque dataset24 (preprocessing code available via GitHub at https://github.com/AdaptiveMotorControlLab/CEBRA). All datasets are publicly accessible without restriction, and no clinical or proprietary third-party data were used in this work.

Code availability

The BCNE code, including the complete pipeline to reproduce all analyses and the preprocessed datasets used in this study, is available via Code Ocean at https://codeocean.com/capsule/3710904/tree (ref. 36). All custom scripts and implementation details supporting the findings of this work are available via GitHub at https://github.com/ZixiaZ/BCNE (ref. 37).

References

  1. Logothetis, N. K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001).

    Article  Google Scholar 

  2. Thakor, N. V. & Tong, S. Advances in quantitative electroencephalogram analysis methods. Annu. Rev. Biomed. Eng. 6, 453–495 (2004).

    Article  Google Scholar 

  3. Bassett, D. S. et al. Dynamic reconfiguration of human brain networks during learning. Proc. Natl Acad. Sci. USA 108, 7641–7646 (2011).

    Article  Google Scholar 

  4. Poldrack, R. A. et al. Interactive memory systems in the human brain. Nature 414, 546–550 (2001).

    Article  Google Scholar 

  5. LeDoux, J. E. Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184 (2000).

    Article  Google Scholar 

  6. Mitra, P. P. & Pesaran, B. Analysis of dynamic brain imaging data. Biophys. J. 76, 691–708 (1999).

    Article  Google Scholar 

  7. Rao, S. M., Mayer, A. R. & Harrington, D. L. The evolution of brain activation during temporal processing. Nat. Neurosci. 4, 317–323 (2001).

    Article  Google Scholar 

  8. Makeig, S. et al. Dynamic brain sources of visual evoked responses. Science 295, 690–694 (2002).

    Article  Google Scholar 

  9. Breakspear, M. Dynamic models of large-scale brain activity. Nat. Neurosci. 20, 340–352 (2017).

    Article  Google Scholar 

  10. Healy, J. & McInnes, L. Uniform manifold approximation and projection. Nat. Rev. Methods Primers 4, 83 (2024).

    Article  Google Scholar 

  11. van der Maaten, L. & Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008).

    Google Scholar 

  12. Moon, K. R. et al. Visualizing structure and transitions in high-dimensional biological data. Nat. Biotechnol. 37, 1482–1492 (2019).

    Article  Google Scholar 

  13. Busch, E. L. et al. Multi-view manifold learning of human brain-state trajectories. Nat. Comput. Sci. 3, 240–253 (2023).

    Article  Google Scholar 

  14. Schneider, S., Lee, J. H. & Mathis, M. W. Learnable latent embeddings for joint behavioural and neural analysis. Nature 617, 360–368 (2023).

    Article  Google Scholar 

  15. Shin, H.-C. et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35, 1285–1298 (2016).

    Article  Google Scholar 

  16. Zhou, Z., Wang, Y., Guo, Y., Qi, Y. & Yu, J. Image quality improvement of hand-held ultrasound devices with a two-stage generative adversarial network. IEEE Trans. Biomed. Eng. 67, 298–311 (2020).

    Article  Google Scholar 

  17. Zhou, Z. et al. Virtual multiplexed immunofluorescence staining from non-antibody-stained fluorescence imaging for gastric cancer prognosis. EBioMedicine 107, 105287 (2024).

    Article  Google Scholar 

  18. Islam, M. T. et al. Revealing hidden patterns in deep neural network feature space continuum via manifold learning. Nat. Commun. 14, 8506 (2023).

    Article  Google Scholar 

  19. Islam, M. T. & Xing, L. Cartography of genomic interactions enables deep analysis of single-cell expression data. Nat. Commun. 14, 679 (2023).

    Article  Google Scholar 

  20. Chen, J. et al. Shared memories reveal shared structure in neural activity across individuals. Nat. Neurosci. 20, 115–125 (2017).

    Article  Google Scholar 

  21. Huang, J. et al. Learning shared neural manifolds from multi-subject fMRI data. In 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing 01–06 (IEEE, 2022); https://doi.org/10.1109/MLSP55214.2022.9943383.

  22. Yates, T. S. et al. Neural event segmentation of continuous experience in human infants. Proc. Natl Acad. Sci. USA 119, e2200257119 (2022).

    Article  Google Scholar 

  23. Baldassano, C. et al. Discovering event structure in continuous narrative perception and memory. Neuron 95, 709–721.e5 (2017).

    Article  Google Scholar 

  24. Chowdhury, R. H., Glaser, J. I. & Miller, L. E. Area 2 of primary somatosensory cortex encodes kinematics of the whole arm. Elife 9, e48198 (2020).

    Article  Google Scholar 

  25. Alonso, I. et al. Peripersonal encoding of forelimb proprioception in the mouse somatosensory cortex. Nat. Commun. 14, 1866 (2023).

    Article  Google Scholar 

  26. Hanin, B. Which neural net architectures give rise to exploding and vanishing gradients? In Advances in Neural Information Processing Systems 582–591 (Curran Associates, Inc., 2018).

  27. Yan, R., Islam, M. T. & Xing, L. Interpretable discovery of patterns in tabular data via spatially semantic topographic maps. Nat. Biomed. Eng. 9, 471–482 (2025).

    Article  Google Scholar 

  28. Vincent, P., Larochelle, H., Bengio, Y. & Manzagol, P.-A. Extracting and composing robust features with denoising autoencoders. In Proc. 25th International Conference on Machine Learning (eds McCallum, A. & Roweis, S. T.) 1096–1103 (Association for Computing Machinery, 2008); https://doi.org/10.1145/1390156.1390294

  29. Asano, Y. M., Rupprecht, C. & Vedaldi, A. Self-labelling via simultaneous clustering and representation learning. In Proc. 8th International Conference on Learning Representations (ICLR 2020) https://openreview.net/pdf?id=Hyx-jyBFPr (2020).

  30. Caron, M., Bojanowski, P., Joulin, A. & Douze, M. Deep clustering for unsupervised learning of visual features. In Computer Vision – ECCV 2018: 15th European Conference, Proceedings, Part XIV (eds V. Ferrari, M. Hebert, C. Sminchisescu & Y. Weiss) 139–156 (Springer, 2018); https://doi.org/10.1007/978-3-030-01264-9_9

  31. Zhou, Z., Zu, X., Wang, Y., Lelieveldt, B. P. F. & Tao, Q. Deep recursive embedding for high-dimensional data. IEEE Trans. Vis. Comput. Graph. 28, 1237–1248 (2022).

    Article  Google Scholar 

  32. Grosmark, A. D. & Buzsáki, G. Diversity in neural firing dynamics supports both rigid and learned hippocampal sequences. Science 351, 1440–1443 (2016).

    Article  Google Scholar 

  33. Pei, F. et al. Neural Latents Benchmark ’21: evaluating latent variable models of neural population activity. In Advances in Neural Information Processing Systems (NeurIPS), Track on Datasets and Benchmarks 34 (Curran Associates, Inc., 2021).

  34. Chen, J. Sherlock movie watching dataset. Princeton University https://doi.org/10.34770/9ndy-8c50 (2016).

  35. Grosmark, A. D., Long, J. & Buzsáki, G. Recordings from hippocampal area CA1, PRE, during and POST novel spatial learning. CRCNS https://doi.org/10.6080/K0862DC5 (2016).

  36. Zhou, Z. BCNE analysis code. Code Ocean https://doi.org/10.24433/CO.3710904.v1 (2025).

  37. Zhou, Z. BCNE analysis code. Zenodo https://doi.org/10.5281/zenodo.16741228 (2025).

Download references

Acknowledgements

This work was partially supported by the 2024 Stanford Human-Centered Artificial Intelligence (HAI) seed grant (L.X. and Z.Z.).

Author information

Authors and Affiliations

  1. Department of Radiation Oncology, Stanford University, Stanford, CA, USA

    Zixia Zhou, Junyan Liu, Wei Emma Wu, Sheng Liu, Qingyue Wei, Rui Yan, Md Tauhidul Islam & Lei Xing

  2. J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA

    Ruogu Fang

  3. Institute of Computational and Mathematical Engineering, Stanford University, Stanford, CA, USA

    Qingyue Wei, Rui Yan & Lei Xing

  4. Department of Electronic Engineering, School of Information Science and Technology, Fudan University, Shanghai, China

    Yi Guo & Yuanyuan Wang

  5. Department of Imaging Physics, Delft University of Technology, Delf, the Netherlands

    Qian Tao

  6. Department of Electrical Engineering, Stanford University, Stanford, CA, USA

    Lei Xing

Authors
  1. Zixia Zhou
    View author publications

    Search author on:PubMed Google Scholar

  2. Junyan Liu
    View author publications

    Search author on:PubMed Google Scholar

  3. Wei Emma Wu
    View author publications

    Search author on:PubMed Google Scholar

  4. Ruogu Fang
    View author publications

    Search author on:PubMed Google Scholar

  5. Sheng Liu
    View author publications

    Search author on:PubMed Google Scholar

  6. Qingyue Wei
    View author publications

    Search author on:PubMed Google Scholar

  7. Rui Yan
    View author publications

    Search author on:PubMed Google Scholar

  8. Yi Guo
    View author publications

    Search author on:PubMed Google Scholar

  9. Qian Tao
    View author publications

    Search author on:PubMed Google Scholar

  10. Yuanyuan Wang
    View author publications

    Search author on:PubMed Google Scholar

  11. Md Tauhidul Islam
    View author publications

    Search author on:PubMed Google Scholar

  12. Lei Xing
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Z.Z., M.T.I. and L.X. conceived and designed the study; Z.Z. established the methodology pipeline and did the statistical analyses; Y.G., Q.T. and Y.W. contributed to refining the methodology; L.X. implemented quality control of data and the algorithms; Z.Z. prepared the first draft of the manuscript; Z.Z., J.L., W.E.W., R.F., S.L., Q.W., R.Y., M.T.I. and L.X. revised the manuscript; All authors contributed to manuscript preparation.

Corresponding authors

Correspondence to Md Tauhidul Islam or Lei Xing.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks Erica Busch, Issam El Naqa, Mohammad Arafat Hussain and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Ananya Rastogi, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 2D Visualizations of the Sherlock fMRI Dataset.

Additional 2D visualizations generated using PCA, t-SNE, UMAP, PHATE, T-PHATE, CEBRA, and BCNE (Recur 0 and Recur 3) across four ROIs. The colormap is the same as in Fig. 2.

Source data

Extended Data Fig. 2 3D Visualizations of the Sherlock fMRI Dataset.

Additional 3D visualizations generated using PCA, t-SNE, UMAP, PHATE, T-PHATE, CEBRA, and BCNE (Recur 0 and Recur 3) across four ROIs. The colormap is the same as in Fig. 2.

Source data

Extended Data Fig. 3 Ablation Experiments on the Sherlock fMRI Dataset.

(a) 2D visualizations for multiple ablation experiments, including BCNE without temporal projection, without spatial projection, with a 1D autoencoder as spatial projection, with a 2D autoencoder as spatial projection, with contrastive loss, and the proposed BCNE, across the EV, HV, EA, and PMV regions. The colormap is the same as in Fig. 2. (b–d) Boxplots, radar maps, and heatmaps displaying KNN classifier accuracy for the 39-event classification task, computed from embeddings produced by the ablation experiments across different regions. KNN accuracies were calculated using a single, randomly selected seed. The same statistical tests and box plot conventions are applied as in Fig. 2.

Source data

Extended Data Fig. 4 Comparative Visualizations and Classification Results.

(a) 3D visualizations of embeddings generated by CEBRA (3D), CEBRA (3D with temporal correlation projection), BCNE (Recur 0), and BCNE (Recur 3). The colormap is the same as in Fig. 2. (b) Boxplots of KNN classifier accuracy calculated from these embeddings. The same statistical tests and box plot conventions are applied as in Fig. 2. (c) Heatmap displaying results of behaviorally-rated event analyses across PCA, t-SNE, UMAP, PHATE, T-PHATE, CEBRA, and BCNE for the four ROIs.

Source data

Extended Data Fig. 5 2D visualizations under varying BCNE configurations and balance parameters.

This figure compares BCNE-generated 2D embeddings of the Sherlock dataset under different architectural configurations (top) and a range of balance-parameter settings (bottom). Each panel shows the low-dimensional trajectory for one of the four ROIs (EA, EV, HV and PMC) at recursion stages 0 and 3. Model-structure experiments evaluate the influence of alternative convolutional and dense-layer designs, while balance-parameter experiments assess the effect of varying the allocation ratio between HD- and LD-manifold components during training. Colormaps follow the same scene-label scheme as in Fig. 2.

Source data

Extended Data Fig. 6 2D Visualizations from BCNE with Different Random Seeds.

Comparison of 2D visualizations generated by the BCNE method with different random seeds across (a) the Sherlock dataset (colormap as in Fig. 2), (b) the rat dataset (colormap as in Fig. 4), and (c) the macaque dataset (colormap as in Fig. 5).

Source data

Extended Data Fig. 7 Ablation and Spatial Projection Analyses for the Rat Hippocampus and Macaque Datasets.

(a) 2D visualizations from ablation experiments on the Rat Hippocampus dataset, including BCNE without temporal projection, without spatial projection, with a 1D autoencoder as spatial projection, with a 2D autoencoder as spatial projection, and the proposed BCNE, shown for four rats. (b) Boxplot of Pearson correlation values between real rat positions and embeddings generated by ablation experiments. (c) Boxplot of KNN classifier accuracy for the learning stage classification task, calculated from ablation embeddings. (d) 2D input data after rearrangement using the compared autocorrelation-based spatial projection, displayed for selected time points (T = 0, 1, 2, 100, 200, 300, 1000, 1008, 1009). (e) 2D input data after rearrangement using the proposed spatial projection of BCNE, shown for the same time points. (f) 2D visualizations generated by BCNE with the compared spatial correlation projection for four rats. (g) Boxplots for Pearson correlation values and KNN classifier accuracy calculated from embeddings generated by the ablation experiments. (h) 2D visualizations from ablation experiments on the Macaque dataset, including BCNE without temporal projection, without spatial projection, with simple spatial correlation projection, with a 1D autoencoder as spatial projection, with a 2D autoencoder as spatial projection, and the proposed BCNE under active or passive modes. The colormap, statistical tests, and box plot conventions are the same as those used in Fig. 4.

Source data

Extended Data Fig. 8 2D Visualizations with Varying Configurations and Balance Parameters on the Rat and Macaque Datasets.

(a) 2D visualizations generated by the BCNE method using the default model and various alternative model configurations on the Rat and Macaque datasets. (b) 2D visualizations generated by the BCNE method with different “balance” parameter settings on the Rat and Macaque datasets (colormap as in Figs. 4 and 5).

Source data

Supplementary information

Source data

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Z., Liu, J., Wu, W.E. et al. Revealing neurocognitive and behavioral patterns through unsupervised manifold learning of dynamic brain data. Nat Comput Sci 5, 1238–1252 (2025). https://doi.org/10.1038/s43588-025-00911-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s43588-025-00911-9

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing