Abstract
Across the primate cortex, neurons with similar functions tend to cluster spatially, a principle that extends across many species and reflects a common strategy for organizing sensory processing. In the visual cortex, this appears as modular clusters tuned to specific visual features. Although short connections are widely believed to support such organization, the underlying neural mechanisms remain unclear. Here, we show that artificial deep neural networks develop topographic maps resembling those in primary, intermediate, and high-level human visual cortex when their units include local lateral connections and are trained through standard top-down credit assignment. Notably, this modular organization emerges without any explicitly imposed topography-inducing objectives or learning rules, suggesting that local lateral connections alone can drive the formation of cortical-like maps. Incorporating such lateral connections also improves model robustness to subtle, adversarial perturbations, highlighting an additional computational role for local recurrent structure in shaping robust visual representations.
Similar content being viewed by others
Data availability
Source data are provided with this paper. The following publicly available resources were used for this work: Imagenet: https://image-net.org/download.php; Natural Scenes Dataset: http://naturalscenesdataset.org. Source data are provided with this paper.
Code availability
We used the following publicly accessible code for building the model and performing the analyses: https://github.com/pytorch/pytorch; https://github.com/neuroailab/TDANN; The code and data associated with the study is publicly accessible at: https://github.com/BashivanLab/LLCNN.
References
Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106 (1962).
Harvey, B. M., Klein, B. P., Petridou, N. & Dumoulin, S. O. Topographic representation of numerosity in the human parietal cortex. Science 341, 1123–1126 (2013).
Humphries, C., Liebenthal, E. & Binder, J. R. Tonotopic organization of human auditory cortex. Neuroimage 50, 1202–1211 (2010).
Obenhaus, H. A. et al. Functional network topography of the medial entorhinal cortex. Proc. Natl. Acad. Sci. 119, e2121655119 (2022).
Gu, Y. et al. A map-like micro-organization of grid cells in the medial entorhinal cortex. Cell 175, 736–750 (2018).
Wong, Y., Kwan, H., MacKay, W. & Murphy, J. Spatial organization of precentral cortex in awake primates. i. somatosensory inputs. J. Neurophysiol. 41, 1107–1119 (1978).
Kanwisher, N., McDermott, J. & Chun, M. M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302–4311 (1997).
Downing, P. E., Jiang, Y., Shuman, M. & Kanwisher, N. A cortical area selective for visual processing of the human body. Science 293, 2470–2473 (2001).
Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392, 598–601 (1998).
Jacobs, R. A. & Jordan, M. I. Computational consequences of a bias toward short connections. J. Cogn. Neurosci. 4, 323–336 (1992).
Kohonen, T. Self-organized formation of topologically correct feature maps. Biol. Cybern. 43, 59–69 (1982).
Von der Malsburg, C. Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100 (1973).
Willshaw, D. J. & Von Der Malsburg, C. How patterned neural connections can be set up by self-organization. Proc. R. Soc. Lond. Ser. B. Biol. Sci. 194, 431–445 (1976).
Lee, H. et al. Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network. BioRxiv https://doi.org/10.1101/2020.07.09.185116 (2020).
Margalit, E. et al. A unifying framework for functional organization in early and higher ventral visual cortex. Neuron 112, 2435–2451 (2024).
Lu, Z., Doerig, A., Bosch, V. et al. End-to-end topographic networks as models of cortical map formation and human visual behaviour. Nat Hum Behav 9, 1975–1991 (2025).
Blauch, N. M., Behrmann, M. & Plaut, D. C. A connectivity-constrained computational account of topographic organization in primate high-level visual cortex. Proc. Natl. Acad. Sci. 119, e2112566119 (2022).
Linsker, R. From basic network principles to neural architecture: emergence of orientation columns. Proc. Natl. Acad. Sci. 83, 8779–8783 (1986).
Durbin, R. & Mitchison, G. A dimension reduction framework for understanding cortical maps. Nature 343, 644–647 (1990).
Doshi, F. R. & Konkle, T. Cortical topographic motifs emerge in a self-organized map of object space. Sci. Adv. 9, eade8187 (2023).
Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).
Horton, J. C. & Hocking, D. R. An adult-like pattern of ocular dominance columns in striate cortex of newborn monkeys prior to visual experience. J. Neurosci. 16, 1791–1807 (1996).
Levay, S., Stryker, M. P. & Shatz, C. J. Ocular dominance columns and their development in layer IV of the cat’s visual cortex: a quantitative study. J. Comp. Neurol. 179, 223–244 (1978).
Blasdel, G. G. & Salama, G. Voltage-sensitive dyes reveal a modular organization in monkey striate cortex. Nature 321, 579–585 (1986).
Smith, G. B., Hein, B., Whitney, D. E., Fitzpatrick, D. & Kaschube, M. Distributed network interactions and their emergence in developing neocortex. Nat. Neurosci. 21, 1600–1608 (2018).
Hubel, D. H. & Wiesel, T. N. Ferrier lecture—functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. Ser. B. Biol. Sci. 198, 1–59 (1997).
Rockland, K. S. & Lund, J. S. Widespread periodic intrinsic connections in the tree shrew visual cortex. Science 215, 1532–1534 (1982).
Rockland, K. S. & Lund, J. S. Intrinsic laminar lattice connections in primate visual cortex. J. Comp. Neurol. 216, 303–318 (1983).
Callaway, E. M. & Katz, L. C. Emergence and refinement of clustered horizontal connections in cat striate cortex. J. Neurosci. 10, 1134–1153 (1990).
Muir, D. R. et al. Embedding of cortical representations by the superficial patch system. Cereb. Cortex 21, 2244–2260 (2011).
Ko, H. et al. Functional specificity of local synaptic connections in neocortical networks. Nature 473, 87–91 (2011).
Ko, H. et al. The emergence of functional microcircuits in visual cortex. Nature 496, 96–100 (2013).
Lund, J. S., Angelucci, A. & Bressloff, P. C. Anatomical substrates for functional columns in macaque monkey primary visual cortex. Cereb. Cortex 13, 15–24 (2003).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).
Walker, E. Y. et al. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci. 22, 2060–2065 (2019).
Kar, K. & Dicarlo, J. J. Fast recurrent processing via ventrolateral prefrontal cortex is needed by the primate ventral stream for robust core visual object recognition. Neuron 109, 164–176.e5 (2021).
Cowley, B. R. et al. Mapping model units to visual neurons reveals population code for social behaviour. Nature 629, 1100–1108 (2024).
Schrimpf, M. et al. Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like? preprint, Neuroscience. http://biorxiv.org/lookup/doi/10.1101/407007 (2018).
Ratan Murty, N. A., Bashivan, P., Abate, A., DiCarlo, J. J. & Kanwisher, N. Computational models of category-selective brain regions enable high-throughput tests of selectivity. Nat. Commun. 12, 5540 (2021).
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A. & Konkle, T. A large-scale examination of inductive biases shaping high-level visual representation in brains and machines. Nat. Commun. 15, 9383 (2024).
Ren, Y. & Bashivan, P. How well do models of visual cortex generalize to out of distribution samples? PLoS Comput. Biol. 20, e1011145 (2024).
Deng, J. et al. Imagenet: a large-scale hierarchical image database. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 248–255 (IEEE, 2009).
Nauhaus, I., Nielsen, K. J., Disney, A. A. & Callaway, E. M. Orthogonal micro-organization of orientation and spatial frequency in primate primary visual cortex. Nat. Neurosci. 15, 1683–1690 (2012).
Marques, T., Schrimpf, M. & DiCarlo, J. J. Multi-scale hierarchical neural network models that bridge from single neurons in the primate primary visual cortex to object recognition behavior. https://doi.org/10.1101/2021.03.01.433495v2 (2021).
Bosking, W. H., Zhang, Y., Schofield, B. & Fitzpatrick, D. Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosci. 17, 2112–2127 (1997).
Schmidt, K. E., Goebel, R., Löwel, S. & Singer, W. The perceptual grouping criterion of colinearity is reflected by anisotropies of connections in the primary visual cortex. Eur. J. Neurosci. 9, 1083–1089 (1997).
Malach, R., Amir, Y., Harel, M. & Grinvald, A. Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. Proc. Natl. Acad. Sci. 90, 10469–10473 (1993).
Bao, P., She, L., McGill, M. & Tsao, D. Y. A map of object space in primate inferotemporal cortex. Nature 583, 103–108 (2020).
Jain, N. et al. Selectivity for food in human ventral visual cortex. Commun. Biol. 6, 1–14 (2023).
Allen, E. J. et al. A massive 7t fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nat. Neurosci. 25, 116–126 (2022).
Nasr, S. et al. Scene-selective cortical regions in human and nonhuman primates. J. Neurosci. 31, 13771–13785 (2011).
Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning 12310–12320 (PMLR, 2021).
Konkle, T. & Oliva, A. A real-world size organization of object responses in occipitotemporal cortex. Neuron 74, 1114–1124 (2012).
Konkle, T. & Caramazza, A. Tripartite organization of the ventral stream by animacy and object size. J. Neurosci. 33, 10235–10242 (2013).
Jiang, R., Andolina, I. M., Li, M. & Tang, S. Clustered functional domains for curves and corners in cortical area v4. eLife 10, e63798 (2021).
Willeke, K. F. et al. Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization. https://doi.org/10.1101/2023.05.12.540591v1 (2023).
Pasupathy, A. & Connor, C. E. Population coding of shape in area V4. Nat. Neurosci. 5, 1332–1338 (2002).
Khosla, M., Ratan Murty, N. A. & Kanwisher, N. A highly selective response to food in human visual cortex revealed by hypothesis-free voxel decomposition. Curr. Biol. https://www.sciencedirect.com/science/article/pii/S0960982222012866 (2022).
Freund, Y. Boosting a weak learning algorithm by majority. Inf. Comput. 121, 256–285 (1995).
Breiman, L. Bagging predictors. Mach. Learn. 24, 123–140 (1996).
Breiman, L. Random forests. Machine Learning. ISBN: 9781441993267 (2001).
Hastie, T., Tibshirani, R., Friedman, J. H. & Friedman, J. H. The Elements of Statistical Learning: Data Mining, Inference, and Prediction Vol. 2 (Springer, 2009).
Croce, F. & Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning 2206–2216 (PMLR, 2020).
Diedrichsen, J. & Kriegeskorte, N. Representational models: a common framework for understanding encoding, pattern-component, and representational-similarity analysis. PLoS Comput. Biol. 13, e1005508 (2017).
Weidler, T. et al. Biologically Inspired Semantic Lateral Connectivity for Convolutional Neural Networks. arXiv https://doi.org/10.48550/arXiv.2105.09830 (2021).
Finzi, D., Margalit, E., Kay, K., Yamins, D. L. K. & Grill-Spector, K. A single computational objective drives specialization of streams in visual cortex. bioRxiv https://doi.org/10.1101/2023.12.19.572460 (2023).
Dehghani, A., Qian, X., Farahani, A. & Bashivan, P. Credit-based self organizing maps: training deep topographic networks with minimal performance degradation. In The Thirteenth International Conference on Learning Representations (2024).
Hyvärinen, A. & Hoyer, P. O. A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vis. Res. 41, 2413–2423 (2001).
Güçlü, U. & Gerven, M. A. J. V. Unsupervised feature learning improves prediction of human brain activity in response to natural images. PLoS Comput. Biol. 10, e1003724 (2014).
Freiwald, W. A. & Tsao, D. Y. Functional compartmentalization and viewpoint generalization within the macaque face-processing system. Science 330, 845–851 (2010).
Martinez, L. M. & Alonso, J.-M. Complex receptive fields in primary visual cortex. Neuroscientist 9, 317–331 (2003).
Chance, F. S., Nelson, S. B. & Abbott, L. Complex cells as cortically amplified simple cells. Nat. Neurosci. 2, 277–282 (1999).
Payeur, A., Guerguiev, J., Zenke, F., Richards, B. & Naud, R. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nat. Neurosci. 24, https://www.nature.com/articles/s41593-021-00857-x (2022).
Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J. & Hinton, G. Backpropagation and the brain. Nat. Rev. Neurosci. 21, 335–346 (2020).
Horton, J. C. & Adams, D. L. The cortical column: a structure without a function. Philos. Trans. R. Soc. B Biol. Sci. 360, 837–862 (2005).
Foerster, O. The motor cortex in man in the light of Hughlings Jackson’s doctrines. Brain 59, 135–159 (1936).
Silver, M. A. & Kastner, S. Topographic maps in human frontal and parietal cortex. Trends Cogn. Sci. 13, 488–495 (2009).
Ross, W. D., Grossberg, S. & Mingolla, E. Visual cortical mechanisms of perceptual grouping: interacting layers, networks, columns, and maps. Neural Netw. 13, 571–588 (2000).
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N. & Kietzmann, T. C. An ecologically motivated image dataset for deep learning yields better models of human vision. Proc. Natl. Acad. Sci. 118, e2011417118 (2021).
Bashivan, P., Bayat, R., Ibrahim, A., Dehghani, A. & Ren, Y. Learning adversarially robust kernel ensembles with kernel average pooling. Expert. Syst. Appl. 266, 126017 (2025).
Boutin, V., Franciosini, A., Chavane, F. & Perrinet, L. U. Pooling strategies in V1 can account for the functional and structural diversity across species. PLoS Comput. Biol. 18, e1010270 (2022).
Arcaro, M. J., McMains, S. A., Singer, B. D. & Kastner, S. Retinotopic organization of human ventral visual cortex. J. Neurosci. 29, 10638–10652 (2009).
Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In The Third International Conference on Learning Representations https://doi.org/10.48550/arXiv.1412.6980 (2014).
Ringach, D. L., Shapley, R. M. & Hawken, M. J. Orientation selectivity in macaque V1: diversity and laminar dependence. J. Neurosci. 22, 5639–5651 (2002).
Walker, E. Y. et al. Inception loops discover what excites neurons most using deep predictive models. Nat Neurosci 22, 2060–2065 (2019).
Stigliani, A., Weiner, K. S. & Grill-Spector, K. Temporal processing capacity in high-level visual cortex is domain specific. J. Neurosci. 35, 12412–12424 (2015).
Claudi, F. Pyramidal neuron. https://zenodo.org/records/3925905; https://doi.org/10.5281/zenodo.3925905 (2020).
Clough, D. Rabbit. https://zenodo.org/records/3926527; https://doi.org/10.5281/zenodo.3926527 (2020).
Acknowledgements
We thank Drs. Nancy Kanwisher, Daniel Yamins, Christopher Pack, Stuart Trenholm, Arjun Krishnaswamy, and Amir Shmuel for their helpful discussions and feedback on the manuscript. We also thank Drs. Pinglei Bao and Doris Tsao for sharing the stimulus set from Bao et al.48 and Dr. Andreas Tolias for sharing the stimulus set from Willeke et al.56. A.O.D. was supported by McGill University’s Health and Science Fellowship. This research was supported by the Healthy-Brains-Healthy-Lives startup supplement grant, the NSERC Discovery grant RGPIN-2021-03035, and CIHR Project Grant PJT-191957. P.B. was supported by FRQ-S Research Scholars Junior 1 grant 310924, FRQNT-NSERC NOVA grant 2024-NOVA-346823, and the William Dawson Scholar award. All analyses were executed using resources provided by the Digital Research Alliance of Canada (Compute Canada) and funding from Canada Foundation for Innovation project number 42730.
Author information
Authors and Affiliations
Contributions
X.Q.: Modeling, Analysis, Software, Validation, Investigation, Writing—Original Draft and Editing. A.O.D.: Data Curation, Investigation—Review and Editing. A.B.F.: Analysis, Validation, Visualization. P.B.: Conceptualization, Methodology, Supervision, Project Administration, Funding Acquisition, Writing—Review and Editing. All authors reviewed and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work. A peer review file is available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Source data
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Qian, X., Dehghani, A.O., Farahani, A.B. et al. Local lateral connectivity is sufficient for replicating cortex-like topographical organization in deep neural networks. Nat Commun (2026). https://doi.org/10.1038/s41467-026-70065-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-026-70065-3
