Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

How can ethology inform the neuroscience of fear, aggression and dominance?

Nature Reviews Neuroscience volume 25, pages 809–819 (2024)Cite this article

Subjects

Abstract

The study of behaviour is dominated by two approaches. On the one hand, ethologists aim to understand how behaviour promotes adaptation to natural contexts. On the other, neuroscientists aim to understand the molecular, cellular, circuit and psychological origins of behaviour. These two complementary approaches must be combined to arrive at a full understanding of behaviour in its natural setting. However, methodological limitations have restricted most neuroscientific research to the study of how discrete sensory stimuli elicit simple behavioural responses under controlled laboratory conditions that are only distantly related to those encountered in real life. Fortunately, the recent advent of neural monitoring and manipulation tools adapted for use in freely behaving animals has enabled neuroscientists to incorporate naturalistic behaviours into their studies and to begin to consider ethological questions. Here, we examine the promises and pitfalls of this trend by describing how investigations of rodent fear, aggression and dominance behaviours are changing to take advantage of an ethological appreciation of behaviour. We lay out current impediments to this approach and propose a framework for the evolution of the field that will allow us to take maximal advantage of an ethological approach to neuroscience and to increase its relevance for understanding human behaviour.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Changing trends in ethological versus non-ethological neuroscience research.

Similar content being viewed by others

Data availability

Data extracted from PubMed to create Fig. 1 is available upon request at gross@embl.it.

References

  1. Tinbergen, N. The Study of Instinct (Oxford Univ. Press, 1951).

  2. Klüver, H. & Bucy, P. C. Preliminary analysis of functions of the temporal lobes in monkeys. Arch. Neurol. Psychiatry 42, 979–1000 (1939).

    Article  Google Scholar 

  3. Hess, W. R. & Brügger, M. Das subkortikale zentrum der affectiven abwehrreactionem [German]. Helv. Physiol. Acta 1, 33–52 (1943).

    Google Scholar 

  4. Canteras, N. S. The medial hypothalamic defensive system: hodological organization and functional implications. Pharmacol. Biochem. Behav. 71, 481–491 (2002).

    Article  CAS  PubMed  Google Scholar 

  5. Gross, C. T. & Canteras, N. S. The many paths to fear. Nat. Rev. Neurosci. 13, 651–658 (2012).

    Article  CAS  PubMed  Google Scholar 

  6. Silva, B. A., Gross, C. T. & Gräff, J. The neural circuits of innate fear: detection, integration, action, and memorization. Learn. Mem. 23, 544–555 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Shaikh, M. B., Brutus, M., Siegel, H. E. & Siegel, A. Topographically organized midbrain modulation of predatory and defensive aggression in the cat. Brain Res. 336, 308–312 (1985).

    Article  CAS  PubMed  Google Scholar 

  8. Gregg, T. R. & Siegel, A. Brain structures and neurotansmitters regulating aggression in cats: implications for human aggression. Prog. Neuropsychopharmacol. Biol. Psychiatry 25, 91–140 (2001).

    Article  CAS  PubMed  Google Scholar 

  9. Swanson, L. W. Anatomy of the soul as reflected in the cerebral hemispheres: neural circuits underlying voluntary control of basic motivated behaviors. J. Comp. Neurol. 493, 122–131 (2005).

    Article  PubMed  Google Scholar 

  10. Siegel, A., Roeling, T. A. P., Gregg, T. R. & Kruk, M. R. Neuropharmacology of brain-stimulation-evoked aggression. Neurosci. Biobehav. Rev. 23, 359–389 (1999).

    Article  CAS  PubMed  Google Scholar 

  11. Gräfe, S. & Stuhrmann, C. Histories of ethology: methods, sites, and dynamics of an unbound discipline. Ber. Zur. Wiss. 45, 10–29 (2022).

    Google Scholar 

  12. Thompson, R. F. Behaviorism and neuroscience. Psychol. Rev. 101, 259–265 (1994).

    Article  CAS  PubMed  Google Scholar 

  13. Blanchard, R. J., Yang, M., Li, C.-I., Gervacio, A. & Blanchard, D. C. Cue and context conditioning of defensive behaviors to cat odor stimuli. Neurosci. Biobehav. Rev. 25, 587–595 (2001).

    Article  CAS  PubMed  Google Scholar 

  14. Ribeiro-Barbosa, E. R., Canteras, N. S., Cezário, A. F., Blanchard, R. J. & Blanchard, D. C. An alternative experimental procedure for studying predator-related defensive responses. Neurosci. Biobehav. Rev. 29, 1255–1263 (2005).

    Article  CAS  PubMed  Google Scholar 

  15. Martinez, R. C., Carvalho-Netto, E. F., Ribeiro-Barbosa, É. R., Baldo, M. V. C. & Canteras, N. S. Amygdalar roles during exposure to a live predator and to a predator-associated context. Neuroscience 172, 314–328 (2011).

    Article  CAS  PubMed  Google Scholar 

  16. Silva, B. A. et al. The ventromedial hypothalamus mediates predator fear memory. Eur. J. Neurosci. 43, 1431–1439 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cavalli, J., Bertoglio, L. J. & Carobrez, A. P. Pentylenetetrazole as an unconditioned stimulus for olfactory and contextual fear conditioning in rats. Neurobiol. Learn. Mem. 92, 512–518 (2009).

    Article  CAS  PubMed  Google Scholar 

  18. Kamikawa, K., McIlwain, J. T. & Adey, W. R. Response patterns of thalamic neurons during classical conditioning. Electroencephalogr. Clin. Neurophysiol. 17, 485–496 (1964).

    Article  CAS  PubMed  Google Scholar 

  19. Carew, T. J., Walters, E. T. & Kandel, E. R. Associative learning in Aplysia: cellular correlates supporting a conditioned fear hypothesis. Science 211, 501–504 (1981).

    Article  CAS  PubMed  Google Scholar 

  20. Applegate, C. D., Frysinger, R. C., Kapp, B. S. & Gallagher, M. Multiple unit activity recorded from amygdala central nucleus during Pavlovian heart rate conditioning in rabbit. Brain Res. 238, 457–462 (1982).

    Article  CAS  PubMed  Google Scholar 

  21. Wurtz, R. H. & Sommer, M. in Methods in Mind (eds Senior, C., Russell, T. & Gazzaniga, M. S.) 123–139 (MIT Press, 2006).

  22. Changeux, J.-P. Discovery of the first neurotransmitter receptor: the acetylcholine nicotinic receptor. Biomolecules 10, 547 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Halliwell, R. F. A short history of the rise of the molecular pharmacology of ionotropic drug receptors. Trends Pharmacol. Sci. 28, 214–219 (2007).

    Article  CAS  PubMed  Google Scholar 

  24. Himanshu, Dharmila, Sarkar, D. & Nutan A review of behavioral tests to evaluate different types of anxiety and anti-anxiety effects. Clin. Psychopharmacol. Neurosci. 18, 341–351 (2020).

    Article  CAS  PubMed Central  Google Scholar 

  25. Lederhendler, I. & Schulkin, J. Behavioral neuroscience: challenges for the era of molecular biology. Trends Neurosci. 23, 451–454 (2000).

    Article  CAS  PubMed  Google Scholar 

  26. Prut, L. & Belzung, C. The open field as a paradigm to measure the effects of drugs on anxiety-like behaviors: a review. Eur. J. Pharmacol. 463, 3–33 (2003).

    Article  CAS  PubMed  Google Scholar 

  27. Bourin, M. Animal models for screening anxiolytic-like drugs: a perspective. Dialogues Clin. Neurosci. 17, 295–303 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Umezu, T. Effects of psychoactive drugs in the Vogel conflict test in mice. Jpn. J. Pharmacol. 80, 111–118 (1999).

    Article  CAS  PubMed  Google Scholar 

  29. Adhikari, A., Topiwala, M. A. & Gordon, J. A. Single units in the medial prefrontal cortex with anxiety-related firing patterns are preferentially influenced by ventral hippocampal activity. Neuron 71, 898–910 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Felix-Ortiz, A. C. et al. BLA to vHPC inputs modulate anxiety-related behaviors. Neuron 79, 658–664 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Tye, K. M. et al. Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature 493, 537–541 (2013).

    Article  CAS  PubMed  Google Scholar 

  32. Olivier, B., Mos, J. & Rasmussen, D. Behavioural pharmacology of the serenic, eltoprazine. Drug. Metabol. Drug. Interact. 8, 31–83 (1990).

    Article  CAS  PubMed  Google Scholar 

  33. Takahashi, A. & Miczek, K. A. Neurogenetics of aggressive behavior — studies in rodents. Curr. Top. Behav. Neurosci. 17, 3–44 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Tulogdi, A. et al. Brain mechanisms involved in predatory aggression are activated in a laboratory model of violent intra-specific aggression. Eur. J. Neurosci. 32, 1744–1753 (2010).

    Article  PubMed  Google Scholar 

  35. Tulogdi, A. et al. Neural mechanisms of predatory aggression in rats — implications for abnormal intraspecific aggression. Behav. Brain Res. 283, 108–115 (2015).

    Article  PubMed  Google Scholar 

  36. Salomon, A. L., Lazorcheck, M. J. & Schein, M. W. Effect of social dominance on individual crowing rates of cockerels. J. Comp. Physiol. Psychol. 61, 144–146 (1966).

    Article  CAS  PubMed  Google Scholar 

  37. Kuse, A. R. & de Fries, J. C. Social dominance and Darwinian fitness in laboratory mice: an alternative test. Behav. Biol. 16, 113–116 (1976).

    Article  CAS  PubMed  Google Scholar 

  38. Chalmers, N. R. & Rowell, T. E. Behaviour and female reproductive cycles in a captive group of mangabeys. Folia Primatol. 14, 1–14 (1971).

    Article  CAS  Google Scholar 

  39. Bernstein, I. S. Dominance: the baby and the bathwater. Behav. Brain Sci. 4, 419–457 (1981).

    Article  Google Scholar 

  40. Francis, R. C. On the relationship between aggression and social dominance. Ethology 78, 223–237 (1988).

    Article  Google Scholar 

  41. Wilson, E. O. Sociobiology: The New Synthesis (Harvard Univ. Press, 1975).

  42. Leander, N. P. & Chartrand, T. L. On thwarted goals and displaced aggression: a compensatory competence model. J. Exp. Soc. Psychol. 72, 88–100 (2017).

    Article  Google Scholar 

  43. Hand, J. L. Resolution of social conflicts: dominance, egalitarianism, spheres of dominance, and game theory. Q. Rev. Biol. 61, 201–220 (1986).

    Article  Google Scholar 

  44. Sapolsky, R. M. The influence of social hierarchy on primate health. Science 308, 648–652 (2005).

    Article  CAS  PubMed  Google Scholar 

  45. Trainor, B. C., Sisk, C. L. & Nelson, R. J. in Hormones, Brain and Behavior Vol. 1 (eds Pfaff, D. W., Arnold, A. P., Etgen, A. M., Fahrbach, S. E. & Rubin, R. T.) 167–203 (Elsevier Academic, 2009).

  46. Lindzey, G., Winston, H. & Manosevitz, M. Social dominance in inbred mouse strains. Nature 191, 474–476 (1961).

    Article  CAS  PubMed  Google Scholar 

  47. Wang, F. et al. Bidirectional control of social hierarchy by synaptic efficacy in medial prefrontal cortex. Science 334, 693–697 (2011).

    Article  CAS  PubMed  Google Scholar 

  48. Fan, Z. et al. Using the tube test to measure social hierarchy in mice. Nat. Protoc. 14, 819–831 (2019).

    Article  CAS  PubMed  Google Scholar 

  49. Battivelli, D. et al. Dopamine neuron activity and stress signaling as links between social hierarchy and psychopathology vulnerability. Biol. Psychiatry 95, 774–784 (2024).

    Article  CAS  PubMed  Google Scholar 

  50. Hou, X. H. et al. Central control circuit for context-dependent micturition. Cell 167, 73–86.e12 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Battivelli, D. et al. Induction of territorial behavior and dominance hierarchies in laboratory mice. Preprint at bioRxiv https://doi.org/10.1101/2024.06.19.599689 (2024).

  52. Zhou, Y. et al. Mice with shank3 mutations associated with ASD and schizophrenia display both shared and distinct defects. Neuron 89, 147–162 (2016).

    Article  CAS  PubMed  Google Scholar 

  53. Strozik, E. & Festing, M. F. Whisker trimming in mice. Lab. Anim. 15, 309–312 (1981).

    Article  CAS  PubMed  Google Scholar 

  54. Zhou, T. et al. History of winning remodels thalamo-PFC circuit to reinforce social dominance. Science 357, 162–168 (2017).

    Article  CAS  PubMed  Google Scholar 

  55. LeClair, K. B. et al. Individual history of winning and hierarchy landscape influence stress susceptibility in mice. eLife 10, e71401 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Padilla-Coreano, N. et al. Cortical ensembles orchestrate social competition through hypothalamic outputs. Nature 603, 667–671 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Li, S. W. et al. Frontal neurons driving competitive behaviour and ecology of social groups. Nature 603, 661–666 (2022).

    Article  CAS  PubMed  Google Scholar 

  58. Zhang, F. et al. Multimodal fast optical interrogation of neural circuitry. Nature 446, 633–639 (2007).

    Article  CAS  PubMed  Google Scholar 

  59. Armbruster, B. N., Li, X., Pausch, M. H., Herlitze, S. & Roth, B. L. Evolving the lock to fit the key to create a family of G protein-coupled receptors potently activated by an inert ligand. Proc. Natl Acad. Sci. USA 104, 5163–5168 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Slimko, E. M., McKinney, S., Anderson, D. J., Davidson, N. & Lester, H. A. Selective electrical silencing of mammalian neurons in vitro by the use of invertebrate ligand-gated chloride channels. J. Neurosci. 22, 7373–7379 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Tsetsenis, T., Ma, X.-H., Lo Iacono, L., Beck, S. G. & Gross, C. Suppression of conditioning to ambiguous cues by pharmacogenetic inhibition of the dentate gyrus. Nat. Neurosci. 10, 896–902 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Jun, J. J. et al. Fully integrated silicon probes for high-density recording of neural activity. Nature 551, 232–236 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Ghosh, K. K. et al. Miniaturized integration of a fluorescence microscope. Nat. Methods 8, 871–878 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Keane, T. M. et al. Mouse genomic variation and its effect on phenotypes and gene regulation. Nature 477, 289–294 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Grubb, S. C., Bult, C. J. & Bogue, M. A. Mouse phenome database. Nucleic Acids Res. 42, D825–D834 (2014).

    Article  CAS  PubMed  Google Scholar 

  66. Doran, A. G. et al. Deep genome sequencing and variation analysis of 13 inbred mouse strains defines candidate phenotypic alleles, private variation and homozygous truncating mutations. Genome Biol. 17, 167 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Hattori, T. et al. Self-exposure to the male pheromone ESP1 enhances male aggressiveness in mice. Curr. Biol. 26, 1229–1234 (2016).

    Article  CAS  PubMed  Google Scholar 

  68. Beck, J. A. et al. Genealogies of mouse inbred strains. Nat. Genet. 24, 23–25 (2000).

    Article  CAS  PubMed  Google Scholar 

  69. Miczek, K. A., Faccidomo, S. P., Fish, E. W. & DeBold, J. F. in Handbook of Neurochemistry and Molecular Neurobiology (eds Lajtha, A. & Blaustein, J. D.) 285–336 (Springer US, 2007).

  70. Bisazza, A. Social organization and territorial behaviour in three strains of mice. Bolletino Zool. 48, 157–167 (1981).

    Article  Google Scholar 

  71. Beans, C. What happens when lab animals go wild. Proc. Natl Acad. Sci. USA 115, 3196–3199 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Canteras, N. S., Chiavegatto, S., Ribeiro do Valle, L. E. & Swanson, L. W. Severe reduction of rat defensive behavior to a predator by discrete hypothalamic chemical lesions. Brain Res. Bull. 44, 297–305 (1997).

    Article  CAS  PubMed  Google Scholar 

  73. Silva, B. A. et al. Independent hypothalamic circuits for social and predator fear. Nat. Neurosci. 16, 1731–1733 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  74. Motta, S. C. et al. Dissecting the brain’s fear system reveals the hypothalamus is critical for responding in subordinate conspecific intruders. Proc. Natl Acad. Sci. USA 106, 4870–4875 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Mongeau, R., Miller, G. A., Chiang, E. & Anderson, D. J. Neural correlates of competing fear behaviors evoked by an innately aversive stimulus. J. Neurosci. 23, 3855–3868 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  76. Rosen, J. B., Asok, A. & Chakraborty, T. The smell of fear: innate threat of 2,5-dihydro-2,4,5-trimethylthiazoline, a single molecule component of a predator odor. Front. Neurosci. 9, 292 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  77. Fendt, M. & Endres, T. 2,3,5-Trimethyl-3-thiazoline (TMT), a component of fox odor — just repugnant or really fear-inducing? Neurosci. Biobehav. Rev. 32, 1259–1266 (2008).

    Article  CAS  PubMed  Google Scholar 

  78. Blanchard, R. J. & Blanchard, D. C. Attack and defense in rodents as ethoexperimental models for the study of emotion. Prog. Neuropsychopharmacol. Biol. Psychiatry 13, S3–S14 (1989).

    Article  PubMed  Google Scholar 

  79. Han, W. et al. Integrated control of predatory hunting by the central nucleus of the amygdala. Cell 168, 311–324.e18 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  80. Carvalho, V. M. et al. Representation of olfactory information in organized active neural ensembles in the hypothalamus. Cell Rep. 32, 108061 (2020).

    Article  CAS  PubMed  Google Scholar 

  81. Yang, T. et al. Social control of hypothalamus-mediated male aggression. Neuron 95, 955–970.e4 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  82. Olszyński, K. H., Polowy, R., Małż, M., Boguszewski, P. M. & Filipkowski, R. K. Playback of alarm and appetitive calls differentially impacts vocal, heart-rate, and motor response in rats. iScience 23, 101577 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Blanchard, R. J. et al. Defensive behaviors in wild and laboratory (Swiss) mice: the mouse defense test battery. Physiol. Behav. 65, 201–209 (1998).

    Article  CAS  PubMed  Google Scholar 

  84. Yilmaz, M. & Meister, M. Rapid innate defensive responses of mice to looming visual stimuli. Curr. Biol. 23, 2011–2015 (2013).

    Article  CAS  PubMed  Google Scholar 

  85. Vale, R., Evans, D. A. & Branco, T. Rapid spatial learning controls instinctive defensive behavior in mice. Curr. Biol. 27, 1342–1349 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  86. Zambetti, P. R. et al. Ecological analysis of Pavlovian fear conditioning in rats. Commun. Biol. 5, 830 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  87. Fadok, J. P. et al. A competitive inhibitory circuit for selection of active and passive fear responses. Nature 542, 96–100 (2017).

    Article  CAS  PubMed  Google Scholar 

  88. Thonhauser, K. E., Raveh, S., Hettyey, A., Beissmann, H. & Penn, D. J. Why do female mice mate with multiple males? Behav. Ecol. Sociobiol. 67, 1961–1970 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  89. Rillich, J., Schildberger, K. & Stevenson, P. A. Octopamine and occupancy: an aminergic mechanism for intruder–resident aggression in crickets. Proc. R. Soc. B Biol. Sci. 278, 1873–1880 (2011).

    Article  Google Scholar 

  90. Crowcroft, P. Mice All Over (Foulis, 1966).

  91. Anderson, P. K. & Hill, J. L. Mus musculus: experimental induction of territory formation. Science 148, 1753–1755 (1965).

    Article  CAS  PubMed  Google Scholar 

  92. Mackintosh, J. H. Territory formation by laboratory mice. Anim. Behav. 18, 177–183 (1970).

    Article  Google Scholar 

  93. Reimer, J. D. & Petras, M. L. Breeding structure of the house mouse, Mus musculus, in a population cage. J. Mammal. 48, 88–99 (1967).

    Article  CAS  PubMed  Google Scholar 

  94. Liu, M., Kim, D.-W., Zeng, H. & Anderson, D. J. Make war not love: the neural substrate underlying a state-dependent switch in female social behavior. Neuron 110, 841–856.e6 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  95. Tovote, P. et al. Midbrain circuits for defensive behaviour. Nature 534, 206–212 (2016).

    Article  CAS  PubMed  Google Scholar 

  96. Wright, K. M. & McDannald, M. A. Ventrolateral periaqueductal gray neurons prioritize threat probability over fear output. eLife 8, e45013 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Lin, D. et al. Functional identification of an aggression locus in the mouse hypothalamus. Nature 470, 221–226 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  98. Carola, V., Scalera, E., Brunamonti, E., Gross, C. & Damato, F. Mating-related interactions share common features with anxiety in the mouse. Behav. Brain Res. 186, 185–190 (2008).

    Article  PubMed  Google Scholar 

  99. Hsu, Y. & Wolf, L. L. The winner and loser effect: integrating multiple experiences. Anim. Behav. 57, 903–910 (1999).

    Article  CAS  PubMed  Google Scholar 

  100. Zhou, T., Sandi, C. & Hu, H. Advances in understanding neural mechanisms of social dominance. Curr. Opin. Neurobiol. 49, 99–107 (2018).

    Article  CAS  PubMed  Google Scholar 

  101. Leyhausen, P. Verhaltensstudien an Katzen [German] (Parey, 1973).

  102. Lindzey, G., Manosevitz, M. & Winston, H. Social dominance in the mouse. Psychon. Sci. 5, 451–452 (1966).

    Article  Google Scholar 

  103. Ahn, S. et al. A role of anterior cingulate cortex in the emergence of worker–parasite relationship. Proc. Natl Acad. Sci. USA 118, e2111145118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  104. Calhoun, J. B. Population density and social pathology. Sci. Am. 206, 139 (1962).

    CAS  PubMed  Google Scholar 

  105. Evans, L. T. Field study of the Social Behavior of the Black Lizard, Ctenosaura pectinata (American Museum of Natural History Library, 1951).

  106. Wang, F., Kessels, H. W. & Hu, H. The mouse that roared: neural mechanisms of social hierarchy. Trends Neurosci. 37, 674–682 (2014).

    Article  CAS  PubMed  Google Scholar 

  107. Anpilov, S. et al. Wireless optogenetic stimulation of oxytocin neurons in a semi-natural setup dynamically elevates both pro-social and agonistic behaviors. Neuron 107, 644–655.e7 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  108. So, N., Franks, B., Lim, S. & Curley, J. P. A social network approach reveals associations between mouse social dominance and brain gene expression. PLoS ONE 10, e0134509 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  109. Sofer, Y. et al. Sexually dimorphic oxytocin circuits drive intragroup social conflict and aggression in wild house mice. Nat. Neurosci. 27, 1565–1573 (2024).

    Article  CAS  PubMed  Google Scholar 

  110. Atasoy, D., Betley, J. N., Su, H. H. & Sternson, S. M. Deconstruction of a neural circuit for hunger. Nature 488, 172–177 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  111. Stagkourakis, S., Spigolon, G., Liu, G. & Anderson, D. J. Experience-dependent plasticity in an innate social behavior is mediated by hypothalamic LTP. Proc. Natl Acad. Sci. USA 117, 25789–25799 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  112. LeDoux, J. E. Anxious: The Modern Mind in the Age of Anxiety (Oneworld Publications, 2015).

  113. Davis, M., Walker, D. L., Miles, L. & Grillon, C. Phasic vs sustained fear in rats and humans: role of the extended amygdala in fear vs anxiety. Neuropsychopharmacology 35, 105–135 (2010).

    Article  PubMed  Google Scholar 

  114. Panksepp, J., Fuchs, T. & Iacobucci, P. The basic neuroscience of emotional experiences in mammals: the case of subcortical FEAR circuitry and implications for clinical anxiety. Appl. Anim. Behav. Sci. 129, 1–17 (2011).

    Article  Google Scholar 

  115. Anderson, D. J. & Adolphs, R. A framework for studying emotions across species. Cell 157, 187–200 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  116. Falkner, A. L. & Lin, D. Recent advances in understanding the role of the hypothalamic circuit during aggression. Front. Syst. Neurosci. 8, 168 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Fatt, P. & Katz, B. Spontaneous subthreshold activity at motor nerve endings. J. Physiol. 117, 109–128 (1952).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  118. Del Castillo, J. & Katz, B. Quantal components of the end‐plate potential. J. Physiol. 124, 560–573 (1954).

    Article  PubMed  PubMed Central  Google Scholar 

  119. Tomko, G. J. & Crapper, D. R. Neuronal variability: non-stationary responses to identical visual stimuli. Brain Res. 79, 405–418 (1974).

    Article  CAS  PubMed  Google Scholar 

  120. Wallace, D. J. et al. Rats maintain an overhead binocular field at the expense of constant fusion. Nature 498, 65–69 (2013).

    Article  CAS  PubMed  Google Scholar 

  121. Steinmetz, N. A. et al. Neuropixels 2.0: a miniaturized high-density probe for stable, long-term brain recordings. Science 372, eabf4588 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  122. Cai, D. J. et al. A shared neural ensemble links distinct contextual memories encoded close in time. Nature 534, 115–118 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  123. Zong, W. et al. Fast high-resolution miniature two-photon microscopy for brain imaging in freely behaving mice. Nat. Methods 14, 713–719 (2017).

    Article  CAS  PubMed  Google Scholar 

  124. Zong, W. et al. Large-scale two-photon calcium imaging in freely moving mice. Cell 185, 1240–1256.e30 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  125. Zong, W. et al. Miniature two-photon microscopy for enlarged field-of-view, multi-plane and long-term brain imaging. Nat. Methods 18, 46–49 (2021).

    Article  CAS  PubMed  Google Scholar 

  126. Klioutchnikov, A. et al. A three-photon head-mounted microscope for imaging all layers of visual cortex in freely moving mice. Nat. Methods 20, 610–616 (2023).

    Article  CAS  PubMed  Google Scholar 

  127. Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).

    Article  CAS  PubMed  Google Scholar 

  128. Kondrakiewicz, K., Kostecki, M., Szadzińska, W. & Knapska, E. Ecological validity of social interaction tests in rats and mice. Genes. Brain Behav. 18, e12525 (2019).

    Article  PubMed  Google Scholar 

  129. Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational neuroethology: a call to action. Neuron 104, 11–24 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  130. Jacobs, B. L. & Fornal, C. A. Activity of serotonergic neurons in behaving animals. Neuropsychopharmacology 21, 9–15 (1999).

    Article  Google Scholar 

  131. Li, Y. et al. Serotonin neurons in the dorsal raphe nucleus encode reward signals. Nat. Commun. 7, 10503 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  132. Luo, M., Li, Y. & Zhong, W. Do dorsal raphe 5-HT neurons encode ‘beneficialness’? Neurobiol. Learn. Mem. 135, 40–49 (2016).

    Article  CAS  PubMed  Google Scholar 

  133. Evans, D. A. et al. A synaptic threshold mechanism for computing escape decisions. Nature 558, 590–594 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  134. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  CAS  PubMed  Google Scholar 

  135. Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2019).

    Article  CAS  PubMed  Google Scholar 

  136. Graving, J. M. et al. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. eLife 8, e47994 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  137. Huang, K. et al. A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping. Nat. Commun. 12, 2784 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  138. Sangiamo, D. T., Warren, M. R. & Neunuebel, J. P. Ultrasonic signals associated with different types of social behavior of mice. Nat. Neurosci. 23, 411–422 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  139. Lauer, J. et al. Multi-animal pose estimation, identification and tracking with DeepLabCut. Nat. Methods 19, 496–504 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  140. Pereira, T. D. et al. SLEAP: a deep learning system for multi-animal pose tracking. Nat. Methods 19, 486–495 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  141. Solié, C., Girard, B., Righetti, B., Tapparel, M. & Bellone, C. VTA dopamine neuron activity encodes social interaction and promotes reinforcement learning through social prediction error. Nat. Neurosci. 25, 86–97 (2022).

    Article  PubMed  Google Scholar 

  142. Mazuski, C. & O’Keefe, J. Representation of ethological events by basolateral amygdala neurons. Cell Rep. 39, 110921 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  143. Abbott, L. F. et al. An international laboratory for systems and computational neuroscience. Neuron 96, 1213–1218 (2017).

    Article  CAS  Google Scholar 

  144. Ayadi, A. et al. Mouse large-scale phenotyping initiatives: overview of the European Mouse Disease Clinic (EUMODIC) and of the Wellcome Trust Sanger Institute Mouse Genetics Project. Mamm. Genome 23, 600–610 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  145. Mallon, A.-M., Blake, A. & Hancock, J. M. EuroPhenome and EMPReSS: online mouse phenotyping resource. Nucleic Acids Res. 36, D715–D718 (2008).

    Article  CAS  PubMed  Google Scholar 

  146. Radvansky, B. A. & Dombeck, D. A. An olfactory virtual reality system for mice. Nat. Commun. 9, 839 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  147. Anderson, D. J. & Perona, P. Toward a science of computational ethology. Neuron 84, 18–31 (2014).

    Article  CAS  PubMed  Google Scholar 

  148. van Daal, R. J. J. et al. Implantation of Neuropixels probes for chronic recording of neuronal activity in freely behaving mice and rats. Nat. Protoc. 16, 3322–3347 (2021).

    Article  PubMed  Google Scholar 

  149. Vogt, N. A bright future for voltage imaging. Nat. Methods 16, 1076–1076 (2019).

    Article  CAS  PubMed  Google Scholar 

  150. Barbera, G., Liang, B., Zhang, L., Li, Y. & Lin, D.-T. A wireless miniScope for deep brain imaging in freely moving mice. J. Neurosci. Methods 323, 56–60 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  151. Shuman, T. et al. Breakdown of spatial coding and interneuron synchronization in epileptic mice. Nat. Neurosci. 23, 229–238 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  152. Wang, Y. et al. Cable-free brain imaging for multiple free-moving animals with miniature wireless microscopes. J. Biomed. Opt. 28, 026503 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  153. Vyssotski, A. L. et al. Long-term monitoring of hippocampus-dependent behavior in naturalistic settings: mutant mice lacking neurotrophin receptor TrkB in the forebrain show spatial learning but impaired behavioral flexibility. Hippocampus 12, 27–38 (2002).

    Article  CAS  PubMed  Google Scholar 

  154. Torquet, N. et al. Social interactions impact on the dopaminergic system and drive individuality. Nat. Commun. 9, 3081 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  155. Williamson, C. M., Lee, W., Romeo, R. D. & Curley, J. P. Social context-dependent relationships between mouse dominance rank and plasma hormone levels. Physiol. Behav. 171, 110–119 (2017).

    Article  CAS  PubMed  Google Scholar 

  156. Zipple, M. N., Vogt, C. C. & Sheehan, M. J. Re-wilding model organisms: opportunities to test causal mechanisms in social determinants of health and aging. Neurosci. Biobehav. Rev. 152, 105238 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  157. Kimchi, T., Xu, J. & Dulac, C. A functional circuit underlying male sexual behaviour in the female mouse brain. Nature 448, 1009–1014 (2007).

    Article  CAS  PubMed  Google Scholar 

  158. Puścian, A. et al. Eco-HAB as a fully automated and ecologically relevant assessment of social impairments in mouse models of autism. eLife 5, e19532 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  159. Shemesh, Y. et al. High-order social interactions in groups of mice. eLife 2, e00759 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  160. Lopez, J. P. et al. Ketamine exerts its sustained antidepressant effects via cell-type-specific regulation of Kcnq2. Neuron 110, 2283–2298.e9 (2022).

    Article  CAS  PubMed  Google Scholar 

  161. Shemesh, Y. et al. Ucn3 and CRF-R2 in the medial amygdala regulate complex social dynamics. Nat. Neurosci. 19, 1489–1496 (2016).

    Article  CAS  PubMed  Google Scholar 

  162. Zilkha, N. et al. Sex-dependent control of pheromones on social organization within groups of wild house mice. Curr. Biol. 33, 1407–1420.e4 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  163. Weissbrod, A. et al. Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment. Nat. Commun. 4, 2018 (2013).

    Article  PubMed  Google Scholar 

  164. Levenstein, D. et al. On the role of theory and modeling in neuroscience. J. Neurosci. 43, 1074–1088 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  165. Flavell, S. W., Gogolla, N., Lovett-Barron, M. & Zelikowsky, M. The emergence and influence of internal states. Neuron 110, 2545–2570 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  166. Viskaitis, P. et al. Modulation of SF1 neuron activity coordinately regulates both feeding behavior and associated emotional states. Cell Rep. 21, 3559–3572 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  167. Waschke, L., Kloosterman, N. A., Obleser, J. & Garrett, D. D. Behavior needs neural variability. Neuron 109, 751–766 (2021).

    Article  CAS  PubMed  Google Scholar 

  168. Levy, D. J., Thavikulwat, A. C. & Glimcher, P. W. State dependent valuation: the effect of deprivation on risk preferences. PLoS ONE 8, e53978 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  169. Burnett, C. J. et al. Hunger-driven motivational state competition. Neuron 92, 187–201 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  170. Whittaker, A. L., Lymn, K. A. & Howarth, G. S. Effects of metabolic cage housing on rat behavior and performance in the social interaction test. J. Appl. Anim. Welf. Sci. 19, 363–374 (2016).

    Article  CAS  PubMed  Google Scholar 

  171. Sofroniew, N. J., Cohen, J. D., Lee, A. K. & Svoboda, K. Natural whisker-guided behavior by head-fixed mice in tactile virtual reality. J. Neurosci. 34, 9537–9550 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  172. Stowers, J. R. et al. Virtual reality for freely moving animals. Nat. Methods 14, 995–1002 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  173. Pinke, D., Issa, J. B., Dara, G. A., Dobos, G. & Dombeck, D. A. Full field-of-view virtual reality goggles for mice. Neuron 111, 3941–3952.e6 (2023).

    Article  CAS  PubMed  Google Scholar 

  174. Thurley, K. Naturalistic neuroscience and virtual reality. Front. Syst. Neurosci. 16, 896251 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  175. Parvizi, J. et al. Complex negative emotions induced by electrical stimulation of the human hypothalamus. Brain Stimul. 15, 615–623 (2022).

    Article  PubMed  Google Scholar 

  176. Wilent, W. B. et al. Induction of panic attack by stimulation of the ventromedial hypothalamus: case report. J. Neurosurg. 112, 1295–1298 (2010).

    Article  PubMed  Google Scholar 

  177. Bugarski-Kirola, D. et al. Efficacy and safety of adjunctive bitopertin versus placebo in patients with suboptimally controlled symptoms of schizophrenia treated with antipsychotics: results from three phase 3, randomised, double-blind, parallel-group, placebo-controlled, multicentre studies in the SearchLyte clinical trial programme. Lancet Psychiatry 3, 1115–1128 (2016).

    Article  PubMed  Google Scholar 

  178. Craven, R. The risky business of drug development in neurology. Lancet Neurol. 10, 116–117 (2011).

    Article  PubMed  Google Scholar 

  179. Steckler, T. & Dautzenberg, F. M. Corticotropin-releasing factor receptor antagonists in affective disorders and drug dependence — an update. CNS Neurol. Disord. Drug. Targets 5, 147–165 (2006).

    Article  CAS  PubMed  Google Scholar 

  180. Alexandrov, V., Brunner, D., Hanania, T. & Leahy, E. High-throughput analysis of behavior for drug discovery. Eur. J. Pharmacol. 750, 82–89 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  181. Vesuna, S. et al. Deep posteromedial cortical rhythm in dissociation. Nature 586, 87–94 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  182. Niepoth, N. & Bendesky, A. How natural genetic variation shapes behavior. Annu. Rev. Genomics Hum. Genet. 21, 437–463 (2020).

    Article  CAS  PubMed  Google Scholar 

  183. Jourjine, N. & Hoekstra, H. E. Expanding evolutionary neuroscience: insights from comparing variation in behavior. Neuron 109, 1084–1099 (2021).

    Article  CAS  PubMed  Google Scholar 

  184. Smith, G. D. & Ebrahim, S. ‘Mendelian randomization’: can genetic epidemiology contribute to understanding environmental determinants of disease? Int. J. Epidemiol. 32, 1–22 (2003).

    Article  PubMed  Google Scholar 

  185. Gasperini, M. et al. A genome-wide framework for mapping gene regulation via cellular genetic screens. Cell 176, 377–390.e19 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  186. Sueur, C., Forin-Wiart, M.-A. & Pelé, M. Are they really trying to save their buddy? The anthropomorphism of animal epimeletic behaviours. Animals 10, 2323 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  187. LeDoux, J. Rethinking the emotional brain. Neuron 73, 653–676 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  188. Panksepp, J. Affective consciousness: core emotional feelings in animals and humans. Conscious. Cogn. 14, 30–80 (2005).

    Article  PubMed  Google Scholar 

  189. Wrangham, R. W. Two types of aggression in human evolution. Proc. Natl Acad. Sci. USA 115, 245–253 (2018).

    Article  CAS  PubMed  Google Scholar 

  190. Wynne, C. D. L. The perils of anthropomorphism. Nature 428, 606–606 (2004).

    Article  CAS  PubMed  Google Scholar 

  191. Wynne, C. D. L. What are animals? Why anthropomorphism is still not a scientific approach to behavior. Comp. Cogn. Behav. Rev. 2, 125–135 (2006).

    Article  Google Scholar 

  192. Burghardt, G. M. Ground rules for dealing with anthropomorphism. Nature 430, 15–15 (2004).

    Article  CAS  PubMed  Google Scholar 

  193. Burghardt, G. M. Critical anthropomorphism, uncritical anthropocentrism, and naïve nominalism. Comp. Cogn. Behav. Rev. 2, 136–138 (2006).

    Article  Google Scholar 

  194. Gomez-Marin, A. A clash of Umwelts: anthropomorphism in behavioral neuroscience. Behav. Brain Sci. 42, e229 (2019).

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Epigenetics & Neurobiology Unit, EMBL Rome, European Molecular Biology Laboratory, Monterotondo, Italy

    Dorian Battivelli & Cornelius T. Gross

  2. School of Brain Science and Brain Medicine, New Cornerstone Science Laboratory, Zhejiang University School of Medicine, Hangzhou, China

    Zhengxiao Fan & Hailan Hu

Authors
  1. Dorian Battivelli
    View author publications

    Search author on:PubMed Google Scholar

  2. Zhengxiao Fan
    View author publications

    Search author on:PubMed Google Scholar

  3. Hailan Hu
    View author publications

    Search author on:PubMed Google Scholar

  4. Cornelius T. Gross
    View author publications

    Search author on:PubMed Google Scholar

Contributions

D.B. and C.T.G. contributed to conceptualization and writing. Z.F. and H.H. contributed to the sections related to dominance and hierarchy.

Corresponding authors

Correspondence to Hailan Hu or Cornelius T. Gross.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Neuroscience thanks David Anderson, who co-reviewed with Kathy Cheung, Jeansok Kim and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

International Society for Neuroethology: https://www.neuroethology.org/

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Battivelli, D., Fan, Z., Hu, H. et al. How can ethology inform the neuroscience of fear, aggression and dominance?. Nat. Rev. Neurosci. 25, 809–819 (2024). https://doi.org/10.1038/s41583-024-00858-2

Download citation

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41583-024-00858-2

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing