Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

A multi-dimensional transfer learning framework for studying reward-guided behaviors across species

Nature Mental Health volume 4, pages 15–29 (2026)Cite this article

Subjects

Abstract

Reward-guided behaviors, essential for survival and adaptation, exhibit both conserved and species-specific features across humans and other animals. Translating findings across species is often hindered by limited cross-species reliability in measurements and uncertain translational validity regarding functional or mechanistic relevance. This Perspective proposes a multi-dimensional transfer learning framework that integrates artificial intelligence (AI) to enhance cross-species research of reward-guided behaviors. By leveraging AI techniques, our framework connects behavioral neuroscience insights from animal models, especially land-based mammals, with functional outcomes in humans, enabling concept- and parameter-level transfer to identify universal principles, clarify mechanisms and optimize experimental paradigms. Using example expression components of behaviors, including locomotion trajectories and facial expressions, we highlight how multi-dimensional transfer learning can reveal conserved neural circuits while accounting for species-specific variations and contextual dynamics. This AI-powered framework offers a promising path to deepen our understanding of reward-guided behaviors and their relevance to mental health disorders.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Conceptual framework for evaluating the cross-species reliability and translational validity of comparable reward-guided behaviors.
Fig. 2: Demonstrations of multi-dimensional characteristics in cross-species reward-guided behavior analysis.

Similar content being viewed by others

References

  1. Fliessbach, K. et al. Social comparison affects reward-related brain activity in the human ventral striatum. Science 318, 1305–1308 (2007).

    Article  PubMed  Google Scholar 

  2. Bore, M. C. et al. Common and separable neural alterations in adult and adolescent depression-evidence from neuroimaging meta-analyses. Neurosci. Biobehav. Rev. 164, 105835 (2024).

    Article  PubMed  Google Scholar 

  3. Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L. & Platt, M. L. A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42, 1931–1939 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Rudebeck, P. H. & Izquierdo, A. Foraging with the frontal cortex: a cross-species evaluation of reward-guided behavior. Neuropsychopharmacology 47, 134–146 (2022).

    Article  PubMed  Google Scholar 

  5. Murray, E. A., O’Doherty, J. P. & Schoenbaum, G. What we know and do not know about the functions of the orbitofrontal cortex after 20 Years of cross-species studies. J. Neurosci. 27, 8166–8169 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Monosov, I. E. Curiosity: primate neural circuits for novelty and information seeking. Nat. Rev. Neurosci. 25, 195–208 (2024).

    Article  PubMed  Google Scholar 

  7. Lewis, A. A non-adaptationist hypothesis of play behaviour. J. Physiol. 602, 2433–2453 (2024).

    Article  PubMed  Google Scholar 

  8. Elias, L. J. & Abdus-Saboor, I. Bridging skin, brain and behavior to understand pleasurable social touch. Curr. Opin. Neurobiol. 73, 102527 (2022).

    Article  PubMed  Google Scholar 

  9. Schiller, D. et al. The human affectome. Neurosci. Biobehav. Rev. 158, 105450 (2024).

    Article  PubMed  Google Scholar 

  10. Lin, F. V., Zuo, Y., Conwell, Y. & Wang, K. H. New horizons in emotional well-being and brain aging: potential lessons from cross-species research. Int. J. Geriatr. Psychiatry 38, e5936 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Ferenczi, E. A. et al. Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior. Science 351, aac9698 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Chandel, A. et al. Thermal infrared directs host-seeking behaviour in Aedes aegypti mosquitoes. Nature 633, 615–623 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  13. Dunbar, R. I. The social role of touch in humans and primates: behavioural function and neurobiological mechanisms. Neurosci. Biobehav. Rev. 34, 260–268 (2010).

    Article  PubMed  Google Scholar 

  14. Barron, H. C., Mars, R. B., Dupret, D., Lerch, J. P. & Sampaio-Baptista, C. Cross-species neuroscience: closing the explanatory gap. Philos. Trans. R. Soc. B Biol. Sci. 376, 20190633 (2020).

    Article  Google Scholar 

  15. Chen, Y. C. I. et al. Detection of dopaminergic neurotransmitter activity using pharmacologic MRI: correlation with PET, microdialysis and behavioral data. Magn. Reson. Med. 38, 389–398 (1997).

    Article  PubMed  Google Scholar 

  16. Patriarchi, T. et al. Ultrafast neuronal imaging of dopamine dynamics with designed genetically encoded sensors. Science 360, eaat4422 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Zürcher, N. R. et al. A simultaneous [11C]raclopride positron emission tomography and functional magnetic resonance imaging investigation of striatal dopamine binding in autism. Transl. Psychiatry 11, 33 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kutlu, M. G. et al. Dopamine release in the nucleus accumbens core signals perceived saliency. Curr. Biol. 31, 4748–4761.e8 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Zachry, J. E. et al. D1 and D2 medium spiny neurons in the nucleus accumbens core have distinct and valence-independent roles in learning. Neuron 112, 835–849.e7 (2024).

    Article  PubMed  Google Scholar 

  20. Ott, T. & Nieder, A. Dopamine and cognitive control in prefrontal cortex. Trends Cogn. Sci. 23, 213–234 (2019).

    Article  PubMed  Google Scholar 

  21. De Neve, J.-E. & Oswald, A. J. Estimating the influence of life satisfaction and positive affect on later income using sibling fixed effects. Proc. Natl Acad. Sci. USA 109, 19953–19958 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Nestler, E. J. & Hyman, S. E. Animal models of neuropsychiatric disorders. Nat. Neurosci. 13, 1161–1169 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Box-Steffensmeier, J. M. et al. The future of human behaviour research. Nat. Hum. Behav. 6, 15–24 (2022).

    Article  PubMed  Google Scholar 

  24. Meng, J. AI emerges as the frontier in behavioral science. Proc. Natl Acad. Sci. USA 121, e2401336121 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Zhao, G., Li, Y. & Xu, Q. From emotion AI to cognitive AI. Int. J. Netw. Dyn. Intell 1, 65–72 (2022).

    Google Scholar 

  26. Lauer, J. et al. Multi-animal pose estimation, identification and tracking with DeepLabCut. Nat. Methods 19, 496–504 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  27. Goodwin, N. L. et al. Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience. Nat. Neurosci. 27, 1411–1424 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Fu, E. Y., Huang, M. X., Leong, H. V. & Ngai, G. Cross-species learning: a low-cost approach to learning human fight from animal fight. In Proc. ACM International Conference on Multimedia 320–327 (ACM, 2018).

  29. Maekawa, T. et al. Cross-species behavior analysis with attention-based domain-adversarial deep neural networks. Nat. Commun. 12, 5519 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006).

    Article  PubMed  Google Scholar 

  31. Berridge, K. C. Reward learning: reinforcement, incentives and expectations. Psychol. Learn. Motiv. 40, 223–278 (2000).

    Article  Google Scholar 

  32. Carver, C. S. & White, T. L. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J. Pers. Soc. Psychol. 67, 319–333 (1994).

    Article  Google Scholar 

  33. Ainslie, G. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol. Bull. 82, 463–496 (1975).

    Article  PubMed  Google Scholar 

  34. Li, S. & Deng, W. Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. 13, 1195–1215 (2020).

    Article  Google Scholar 

  35. Kong, Y. & Fu, Y. Human action recognition and prediction: a survey. Int. J. Comput. Vis. 130, 1366–1401 (2022).

    Article  Google Scholar 

  36. Can, Y. S., Mahesh, B. & André, E. Approaches, applications, and challenges in physiological emotion recognition—a tutorial overview. Proc. IEEE 111, 1287–1313 (2023).

    Article  Google Scholar 

  37. Lindgren, H. Emerging roles and relationships among humans and interactive AI systems. Int. J. Human Comput. Interact. 41, 10595–10617 (2024).

    Article  Google Scholar 

  38. Niedenthal, P. M., Mermillod, M., Maringer, M. & Hess, U. The Simulation of Smiles (SIMS) model: embodied simulation and the meaning of facial expression. Behav. Brain Sci. 33, 417–433 (2010).

    Article  PubMed  Google Scholar 

  39. Cowen, A. S. & Keltner, D. What the face displays: mapping 28 emotions conveyed by naturalistic expression. Am. Psychol. 75, 349–364 (2020).

    Article  PubMed  Google Scholar 

  40. Cowen, A. S. et al. Sixteen facial expressions occur in similar contexts worldwide. Nature 589, 251–257 (2021).

    Article  PubMed  Google Scholar 

  41. Wang, Y. et al. Vision-based estimation of fatigue and engagement in cognitive training sessions. Artif. Intell. Med. 154, 102923 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Awais, M. et al. A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video. IEEE J. Biomed. Health Inform. 25, 1441–1449 (2021).

    Article  PubMed  Google Scholar 

  43. Savchenko, A. V., Savchenko, L. V. & Makarov, I. Classifying emotions and engagement in online learning based on a single facial expression recognition neural network. IEEE Trans. Affect. Comput. 13, 2132–2143 (2022).

    Article  Google Scholar 

  44. Kim, E., Bryant, D., Srikanth, D. & Howard, A. Age bias in emotion detection: an analysis of facial emotion recognition performance on young, middle-aged, and older adults. In Proc. AAAI/ACM Conference on AI, Ethics and Society 638–644 (ACM, 2021).

  45. Li, Y., Wei, J., Liu, Y., Kauttonen, J. & Zhao, G. Deep learning for micro-expression recognition: a survey. IEEE Trans. Affect. Comput. 13, 2028–2046 (2022).

    Article  Google Scholar 

  46. Veltmeijer, E. A., Gerritsen, C. & Hindriks, K. V. Automatic emotion recognition for groups: a review. IEEE Trans. Affect. Comput. 14, 89–107 (2023).

    Article  Google Scholar 

  47. Walawalkar, D. & Garrido, P. VideoClusterNet: self-supervised and adaptive face clustering for videos. In Proc. European Conference on Computer Vision 377–396 (Springer, 2024).

  48. Cyders, M. A. & Smith, G. T. Emotion-based dispositions to rash action: positive and negative urgency. Psychol. Bull. 134, 807–828 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Chou, K.-P., Wilson, R. C. & Smith, R. The influence of anxiety on exploration: a review of computational modeling studies. Neurosci. Biobehav. Rev. 167, 105940 (2024).

    Article  PubMed  Google Scholar 

  50. Song, S. et al. Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation. J. Neuroeng. Rehabil. 18, 126 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  51. Endo, M. et al. Data-driven discovery of movement-linked heterogeneity in neurodegenerative diseases. Nat. Mach. Intell. 6, 1034–1045 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Liu, X. et al. iMiGUE: an identity-free video dataset for micro-gesture understanding and emotion analysis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10631–10642 (IEEE, 2021).

  53. Chen, H., Shi, H., Liu, X., Li, X. & Zhao, G. SMG: a micro-gesture dataset towards spontaneous body gestures for emotional stress state analysis. Int. J. Comput. Vis. 131, 1346–1366 (2023).

    Article  Google Scholar 

  54. Logge, W. B., Morley, K. C., Haber, P. S. & Baillie, A. J. Impaired decision-making and skin conductance responses are associated with reward and punishment sensitivity in individuals with severe alcohol use disorder. Neuropsychobiology 82, 117–129 (2023).

    Article  PubMed  Google Scholar 

  55. Fouragnan, E. F. et al. Timing along the cardiac cycle modulates neural signals of reward-based learning. Nat. Commun. 15, 2976 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  56. Lin, F. V. & Heffner, K. L. Autonomic nervous system flexibility for understanding brain aging. Ageing Res. Rev. 90, 102016 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  57. Sun, Z. & Li, X. Contrast-Phys+: unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal contrast. IEEE Trans. Pattern Anal. Mach. Intell. 46, 5835–5851 (2024).

    Article  PubMed  Google Scholar 

  58. Lu, Y. & Zhong, S. Contactless real-time heart rate predicts the performance of elite athletes: evidence from Tokyo 2020 Olympic archery competition. Psychol. Sci. 34, 384–393 (2023).

    Article  PubMed  Google Scholar 

  59. Poremba, A., Bigelow, J. & Rossi, B. Processing of communication sounds: contributions of learning, memory and experience. Hear. Res. 305, 31–44 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Massaccesi, C., Korb, S., Skoluda, N., Nater, U. M. & Silani, G. Effects of appetitive and aversive motivational states on wanting and liking of interpersonal touch. Neuroscience 464, 12–25 (2021).

    Article  PubMed  Google Scholar 

  61. Zhang, Z., Cheng, H. & Yang, T. A recurrent neural network framework for flexible and adaptive decision making based on sequence learning. PLoS Comput. Biol. 16, e1008342 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Lian, Z. et al. AffectGPT: a new dataset, model and benchmark for emotion understanding with multimodal large language models. In Proc. International Conference on Machine Learning 267, 36993–37014 (PMLR, 2025).

  63. Mihalcea, R. et al. How developments in natural language processing help us in understanding human behaviour. Nat. Hum. Behav. 8, 1877–1889 (2024).

    Article  PubMed  Google Scholar 

  64. Bain, M. et al. Automated audiovisual behavior recognition in wild primates. Sci. Adv. 7, eabi4883 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  65. Maekawa, T. et al. Deep learning-assisted comparative analysis of animal trajectories with DeepHL. Nat. Commun. 11, 5316 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  66. Kain, J. et al. Leg-tracking and automated behavioural classification in Drosophila. Nat. Commun. 4, 1910 (2013).

    Article  PubMed  Google Scholar 

  67. Chen, J. et al. MammalNet: a large-scale video benchmark for mammal recognition and behavior understanding. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 13052–13061 (IEEE, 2023).

  68. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  PubMed  Google Scholar 

  69. Romero-Ferrero, F., Bergomi, M. G., Hinz, R. C., Heras, F. J. H. & de Polavieja, G. G. idtracker.ai: tracking all individuals in small or large collectives of unmarked animals. Nat. Methods 16, 179–182 (2019).

    Article  PubMed  Google Scholar 

  70. Couzin, I. D. & Heins, C. Emerging technologies for behavioral research in changing environments. Trends Ecol. Evol. 38, 346–354 (2023).

    Article  PubMed  Google Scholar 

  71. Stijovic, A. et al. Defining social reward: a systematic review of human and animal studies. Psychol. Bull. 150, 1472–1509 (2024).

    Article  PubMed  Google Scholar 

  72. Khan, M. H. et al. AnimalWeb: a large-scale hierarchical dataset of annotated animal faces. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 6939–6948 (IEEE, 2020).

  73. Boneh-Shitrit, T. et al. Explainable automated recognition of emotional states from canine facial expressions: the case of positive anticipation and frustration. Sci. Rep. 12, 22611 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Carmo, L. G., Werner, L. C., Michelotto, P. V. Jr & Daros, R. R. Horse behavior and facial movements in relation to food rewards. PLoS ONE 18, e0286045 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  75. Syeda, A. et al. Facemap: a framework for modeling neural activity based on orofacial tracking. Nat. Neurosci. 27, 187–195 (2024).

    Article  PubMed  Google Scholar 

  76. Pessanha, F., McLennan, K. & Mahmoud, M. Towards automatic monitoring of disease progression in sheep: a hierarchical model for sheep facial expressions analysis from video. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition 387–393 (IEEE, 2020).

  77. Pessanha, F., Salah, A. A., van Loon, T. & Veltkamp, R. Facial image-based automatic assessment of equine pain. IEEE Trans. Affect. Comput. 14, 2064–2076 (2022).

    Article  Google Scholar 

  78. Martvel, G., Shimshoni, I. & Zamansky, A. Automated detection of cat facial landmarks. Int. J. Comput. Vis. 132, 3103–3118 (2024).

    Article  Google Scholar 

  79. Waidmann, E. N., Koyano, K. W., Hong, J. J., Russ, B. E. & Leopold, D. A. Local features drive identity responses in macaque anterior face patches. Nat. Commun. 13, 5592 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  80. Yao, Y. et al. OpenMonkeyChallenge: dataset and benchmark challenges for pose estimation of non-human primates. Int. J. Comput. Vis. 131, 243–258 (2023).

    Article  PubMed  Google Scholar 

  81. Shooter, M., Malleson, C. & Hilton, A. SyDog-Video: a synthetic dog video dataset for temporal pose estimation. Int. J. Comput. Vis. 132, 1986–2002 (2024).

    Article  Google Scholar 

  82. Marks, M. et al. Deep-learning-based identification, tracking, pose estimation and behaviour classification of interacting primates and mice in complex environments. Nat. Mach. Intell. 4, 331–340 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Shi, X., Yang, C., Xia, X. & Chai, X. Deep cross-species feature learning for animal face recognition via residual interspecies equivariant network. In Proc. European Conference on Computer Vision 667–682 (Springer, 2020).

  84. Zhang, H. et al. Equivalent processing of facial expression and identity by macaque visual system and task-optimized neural network. NeuroImage 273, 120067 (2023).

    Article  PubMed  Google Scholar 

  85. Kwon, J. et al. SUBTLE: an unsupervised platform with temporal link embedding that maps animal behavior. Int. J. Comput. Vis. 132, 4589–4615 (2024).

    Article  Google Scholar 

  86. Shaw, L., Wang, K. H. & Mitchell, J. Fast prediction in marmoset reach-to-grasp movements for dynamic prey. Curr. Biol. 33, 2557–2565.e4 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  87. Pereira, T. D. et al. SLEAP: a deep learning system for multi-animal pose tracking. Nat. Methods 19, 486–495 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  88. Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2019).

    Article  PubMed  Google Scholar 

  89. Dunn, T. W. et al. Geometric deep learning enables 3D kinematic profiling across species and environments. Nat. Methods 18, 564–573 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  90. Nathan, R. et al. Big-data approaches lead to an increased understanding of the ecology of animal movement. Science 375, eabg1780 (2022).

    Article  PubMed  Google Scholar 

  91. Broomé, S. et al. Going deeper than tracking: a survey of computer-vision based recognition of animal pain and emotions. Int. J. Comput. Vis. 131, 572–590 (2023).

    Article  Google Scholar 

  92. Kaneko, T. et al. Deciphering social traits and pathophysiological conditions from natural behaviors in common marmosets. Curr. Biol. 34, 2854–2867.e5 (2024).

    Article  PubMed  Google Scholar 

  93. Cai, X. et al. Dopamine dynamics are dispensable for movement but promote reward responses. Nature 635, 406–414 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  94. Peysakhovich, B. et al. Primate superior colliculus is causally engaged in abstract higher-order cognition. Nat. Neurosci. 27, 1999–2008 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  95. Hsueh, B. et al. Cardiogenic control of affective behavioural state. Nature 615, 292–299 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  96. Henry, B. L. et al. Cross-species assessments of motor and exploratory behavior related to bipolar disorder. Neurosci. Biobehav. Rev. 34, 1296–1306 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Fan, S., Monte, O. D., Nair, A. R., Fagan, N. A. & Chang, S. W. C. Closed-loop microstimulations of the orbitofrontal cortex during real-life gaze interaction enhance dynamic social attention. Neuron 112, 2631–2644.e6 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Skora, L., Livermore, J. & Roelofs, K. The functional role of cardiac activity in perception and action. Neurosci. Biobehav. Rev. 137, 104655 (2022).

    Article  PubMed  Google Scholar 

  99. Grujic, N., Polania, R. & Burdakov, D. Neurobehavioral meaning of pupil size. Neuron 112, 3381–3395 (2024).

    Article  PubMed  Google Scholar 

  100. Chang, S. W. C. et al. Neuroethology of primate social behavior. Proc. Natl Acad. Sci. USA 110, 10387–10394 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  101. McFarland, R. et al. Social integration confers thermal benefits in a gregarious primate. J. Anim. Ecol. 84, 871–878 (2015).

    Article  PubMed  Google Scholar 

  102. Mitchell, J. F., Wang, K. H., Batista, A. P. & Miller, C. T. An ethologically motivated neurobiology of primate visually-guided reach-to-grasp behavior. Curr. Opin. Neurobiol. 86, 102872 (2024).

    Article  PubMed  Google Scholar 

  103. Gábor, A. et al. Social relationship-dependent neural response to speech in dogs. Neuroimage 243, 118480 (2021).

    Article  PubMed  Google Scholar 

  104. Fonseca, A. H., Santana, G. M., Bosque Ortiz, G. M., Bampi, S. & Dietrich, M. O. Analysis of ultrasonic vocalizations from mice using computer vision and machine learning. eLife 10, e59161 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  105. Jung, D.-H. et al. Deep learning-based cattle vocal classification model and real-time livestock monitoring system with noise filtering. Animals 11, 357 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  106. Zeng, D., Hong, S., Li, S., Shen, Q. & Tang, B. Data-scarce animal face alignment via bi-directional cross-species knowledge transfer. In Proc. ACM International Conference on Multimedia 8475–8485 (ACM, 2023).

  107. Ekman, P., Friesen, W. V. & Hager, J. C. Facial Action Coding System: The Manual (Research Nexus, 2002).

  108. Parr, L. A., Waller, B. M., Burrows, A. M., Gothard, K. M. & Vick, S.-J. Brief communication: MaqFACS: a muscle-based facial movement coding system for the rhesus macaque. Am. J. Phys. Anthropol. 143, 625–630 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  109. Correia-Caeiro, C., Burrows, A., Wilson, D. A., Abdelrahman, A. & Miyabe-Nishiwaki, T. CalliFACS: the common marmoset Facial Action Coding System. PLoS ONE 17, e0266442 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  110. Waller, B. M. et al. Paedomorphic facial expressions give dogs a selective advantage. PLoS ONE 8, e82686 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  111. Caeiro, C. C., Burrows, A. M. & Waller, B. M. Development and application of CatFACS: are human cat adopters influenced by cat facial expressions?. Appl. Anim. Behav. Sci. 189, 66–78 (2017).

    Article  Google Scholar 

  112. Waller, B. M., Julle-Daniere, E. & Micheletta, J. Measuring the evolution of facial ‘expression’ using multi-species FACS. Neurosci. Biobehav. Rev. 113, 1–11 (2020).

    Article  PubMed  Google Scholar 

  113. Nieuwburg, E. G., Ploeger, A. & Kret, M. E. Emotion recognition in nonhuman primates: how experimental research can contribute to a better understanding of underlying mechanisms. Neurosci. Biobehav. Rev. 123, 24–47 (2021).

    Article  PubMed  Google Scholar 

  114. Sosa, M. & Giocomo, L. M. Navigating for reward. Nat. Rev. Neurosci. 22, 472–487 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  115. Hunt, L. et al. Formalizing planning and information search in naturalistic decision-making. Nat. Neurosci. 24, 1051–1064 (2021).

    Article  PubMed  Google Scholar 

  116. Artoni, P. et al. Deep learning of spontaneous arousal fluctuations detects early cholinergic defects across neurodevelopmental mouse models and patients. Proc. Natl Acad. Sci. USA 117, 23298–23303 (2020).

    Article  PubMed  Google Scholar 

  117. Martins, P. T. & Boeckx, C. Vocal learning: beyond the continuum. PLoS Biol. 18, e3000672 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  118. Ravignani, A. et al. Rhythm in speech and animal vocalizations: a cross-species perspective. Ann. N. Y. Acad. Sci. 1453, 79–98 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  119. Gabeff, V., Rußwurm, M., Tuia, D. & Mathis, A. WildCLIP: scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models. Int. J. Comput. Vis. 32, 3770–3786 (2024).

    Article  Google Scholar 

  120. Diester, I. et al. Internal world models in humans, animals, and AI. Neuron 112, 2265–2268 (2024).

    Article  PubMed  Google Scholar 

  121. Averbeck, B. B. & Costa, V. D. Motivational neural circuits underlying reinforcement learning. Nat. Neurosci. 20, 505–512 (2017).

    Article  PubMed  Google Scholar 

  122. Ye, S. et al. SuperAnimal pretrained pose estimation models for behavioral analysis. Nat. Commun. 15, 5165 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  123. Brookes, O. et al. PanAf20K: a large video dataset for wild ape detection and behaviour recognition. Int. J. Comput. Vis. 132, 3086–3102 (2024).

    Article  Google Scholar 

  124. Ye, S., Lauer, J., Zhou, M., Mathis, A. & Mathis, M. AmadeusGPT: a natural language interface for interactive animal behavioral analysis. Adv. Neural Inf. Process. Syst. 36, 6297–6329 (2023).

    Google Scholar 

  125. Ding, X. & Zhang, H. Dissociation and hierarchy of human visual pathways for simultaneously coding facial identity and expression. NeuroImage 264, 119769 (2022).

    Article  PubMed  Google Scholar 

  126. Driess, D. et al. Palm-e: An embodied multimodal language model. In Proc. International Conference on Machine Learning 202, 8469–8488 (PMLR, 2023).

  127. Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. International Conference on Machine Learning 139, 8748–8763 (PMLR, 2021).

  128. Zhang, Y. et al. Exploring the transferability of visual prompting for multimodal large language models. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 26562–26572 (IEEE, 2024).

  129. Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C. & Jorge, J. Counterfactuals and causability in explainable artificial intelligence: theory, algorithms and applications. Inf. Fusion 81, 59–83 (2022).

    Article  Google Scholar 

  130. Jaimini, U. & Sheth, A. CausalKG: Causal Knowledge Graph explainability using interventional and counterfactual reasoning. IEEE Internet Comput. 26, 43–50 (2022).

    Article  Google Scholar 

  131. Feuerriegel, S. et al. Causal machine learning for predicting treatment outcomes. Nat. Med. 30, 958–968 (2024).

    Article  PubMed  Google Scholar 

  132. Prosperi, M. et al. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nat. Mach. Intell. 2, 369–375 (2020).

    Article  Google Scholar 

  133. Rozantsev, A., Salzmann, M. & Fua, P. Residual parameter transfer for deep domain adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4339–4348 (IEEE, 2018).

  134. Kanakasabapathy, M. K. et al. Adaptive adversarial neural networks for the analysis of lossy and domain-shifted datasets of medical images. Nat. Biomed. Eng. 5, 571–585 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  135. Subramanian, S. et al. Towards foundation models for scientific machine learning: characterizing scaling and transfer behavior. In Proc. International Conference on Neural Information Processing Systems 36, 71242–71262 (Curran Associates, 2024).

  136. Jiang, S. et al. Confounder balancing in adversarial domain adaptation for pre-trained large models fine-tuning. Neural Netw. 173, 106173 (2024).

    Article  PubMed  Google Scholar 

  137. Luxem, K. et al. Identifying behavioral structure from deep variational embeddings of animal motion. Commun. Biol. 5, 1267 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  138. Choi, S. H. et al. Individual variations lead to universal and cross-species patterns of social behavior. Proc. Natl Acad. Sci. USA 117, 31754–31759 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  139. Kaiser, M. I., Gadau, J., Kaiser, S., Müller, C. & Richter, S. H. Individualized social niches in animals: theoretical clarifications and processes of niche change. BioScience 74, 146–158 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  140. Rocha, L. E., Ryckebusch, J., Schoors, K. & Smith, M. The scaling of social interactions across animal species. Sci. Rep. 11, 12584 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  141. Schino, G. & Pinzaglia, M. Age-related changes in the social behavior of tufted capuchin monkeys. Am. J. Primatol. 80, e22746 (2018).

    Article  PubMed  Google Scholar 

  142. Zhang, T.-Y. et al. Brain-derived neurotrophic factor in the nucleus accumbens mediates individual differences in behavioral responses to a natural, social reward. Mol. Neurobiol. 57, 290–301 (2020).

    Article  PubMed  Google Scholar 

  143. Cavanagh, J. F. et al. Electrophysiological biomarkers of behavioral dimensions from cross-species paradigms. Transl. Psychiatry 11, 482 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  144. Brynildsen, J. K., Rajan, K., Henderson, M. X. & Bassett, D. S. Network models to enhance the translational impact of cross-species studies. Nat. Rev. Neurosci. 24, 575–588 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  145. Devinsky, O. et al. A cross-species approach to disorders affecting brain and behaviour. Nat. Rev. Neurol. 14, 677–686 (2018).

    Article  PubMed  Google Scholar 

  146. Kokkinou, M. et al. Reproducing the dopamine pathophysiology of schizophrenia and approaches to ameliorate it: a translational imaging study with ketamine. Mol. Psychiatry 26, 2562–2576 (2021).

    Article  PubMed  Google Scholar 

  147. Salvan, P. et al. Serotonin regulation of behavior via large-scale neuromodulation of serotonin receptor networks. Nat. Neurosci. 26, 53–63 (2023).

    Article  PubMed  Google Scholar 

  148. Gyles, T. M., Nestler, E. J. & Parise, E. M. Advancing preclinical chronic stress models to promote therapeutic discovery for human stress disorders. Neuropsychopharmacology 49, 215–226 (2024).

    Article  PubMed  Google Scholar 

  149. Shemesh, Y. & Chen, A. A paradigm shift in translational psychiatry through rodent neuroethology. Mol. Psychiatry 28, 993–1003 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  150. Nani, J. V., Muotri, A. R. & Hayashi, M. A. F. Peering into the mind: unraveling schizophrenia’s secrets using models. Mol. Psychiatry 30, 659–678 (2025).

    Article  PubMed  Google Scholar 

  151. Yang, Z. & Long, M. A. Convergent vocal representations in parrot and human forebrain motor networks. Nature 640, 427–434 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  152. Weinreb, C. et al. Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics. Nat. Methods 21, 1329–1339 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  153. Flórez-Vargas, O. et al. Bias in the reporting of sex and age in biomedical research on mouse models. eLife 5, e13615 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  154. Rosenbacke, R., Melhus, Å, McKee, M. & Stuckler, D. How explainable artificial intelligence can increase or decrease clinicians’ trust in AI applications in health care: systematic review. JMIR AI 3, e53207 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  155. Hassija, V. et al. Interpreting black-box models: a review on explainable artificial intelligence. Cogn. Comput. 16, 45–74 (2024).

    Article  Google Scholar 

  156. Hagendorff, T. The ethics of AI ethics: an evaluation of guidelines. Minds Mach. 30, 99–120 (2020).

    Article  Google Scholar 

  157. Ienca, M., Jotterand, F., Vică, C. & Elger, B. Social and assistive robotics in dementia care: ethical recommendations for research and practice. Int. J. Soc. Robot. 8, 565–573 (2016).

    Article  Google Scholar 

  158. Herlin, A. et al. Animal welfare implications of digital tools for monitoring and management of cattle and sheep on pasture. Animals 11, 829 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  159. Cartmill, E. A. Overcoming bias in the comparison of human language and animal communication. Proc. Natl Acad. Sci. USA 120, e2218799120 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  160. Sakamaki, T. Social grooming among wild bonobos (Pan paniscus) at Wamba in the Luo Scientific Reserve, DR Congo, with special reference to the formation of grooming gatherings. Primates 54, 349–359 (2013).

    Article  PubMed  Google Scholar 

  161. Bigiani, S. & Pilenga, C. Cooperation increases bottlenose dolphins’ (Tursiops truncatus) social affiliation. Anim. Cogn. 26, 1319–1333 (2023).

    Article  PubMed  Google Scholar 

  162. Riters, L. V. Pleasure seeking and birdsong. Neurosci. Biobehav. Rev. 35, 1837–1845 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  163. Siniscalchi, M., Lusito, R., Vallortigara, G. & Quaranta, A. Seeing left-or right-asymmetric tail wagging produces different emotional responses in dogs. Curr. Biol. 23, 2279–2282 (2013).

    Article  PubMed  Google Scholar 

  164. Humphrey, T., Stringer, F., Proops, L. & McComb, K. Slow blink eye closure in shelter cats is related to quicker adoption. Animals 10, 2256 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  165. Gainotti, G. Hemispheric asymmetries for emotions in non-human primates: a systematic review. Neurosci. Biobehav. Rev. 141, 104830 (2022).

    Article  PubMed  Google Scholar 

  166. Lee, S. M. et al. Wild bonobo and chimpanzee females exhibit broadly similar patterns of behavioral maturation but some evidence for divergence. Am. J. Phys. Anthropol. 171, 100–109 (2020).

    Article  PubMed  Google Scholar 

  167. Palagi, E. Adult play and the evolution of tolerant and cooperative societies. Neurosci. Biobehav. Rev. 148, 105124 (2023).

    Article  PubMed  Google Scholar 

  168. Numan, M. & Young, L. J. Neural mechanisms of mother-infant bonding and pair bonding: similarities, differences and broader implications. Horm. Behav. 77, 98–112 (2016).

    Article  PubMed  Google Scholar 

  169. Kret, M. E., Prochazkova, E., Sterck, E. H. M. & Clay, Z. Emotional expressions in human and non-human great apes. Neurosci. Biobehav. Rev. 115, 378–395 (2020).

    Article  PubMed  Google Scholar 

  170. Sueur, C. & Huffman, M. A. Co-cultures: exploring interspecies culture among humans and other animals. Trends Ecol. Evol. 39, 821–829 (2024).

    Article  PubMed  Google Scholar 

  171. Stangl, M., Maoz, S. L. & Suthana, N. Mobile cognition: imaging the human brain in the ‘real world’. Nat. Rev. Neurosci. 24, 347–362 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  172. Kalan, A. K. et al. Environmental variability supports chimpanzee behavioural diversity. Nat. Commun. 11, 4451 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  173. Anderson, J. R., Ang, M. Y., Lock, L. C. & Weiche, I. Nesting, sleeping and nighttime behaviors in wild and captive great apes. Primates 60, 321–332 (2019).

    Article  PubMed  Google Scholar 

  174. Shultz, S. & Dunbar, R. I. Socioecological complexity in primate groups and its cognitive correlates. Philos. Trans. R. Soc. B 377, 20210296 (2022).

    Article  Google Scholar 

  175. Kong, E., Lee, K.-H., Do, J., Kim, P. & Lee, D. Dynamic and stable hippocampal representations of social identity and reward expectation support associative social memory in male mice. Nat. Commun. 14, 2597 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  176. Cait, J., Cait, A., Scott, R. W., Winder, C. B. & Mason, G. J. Conventional laboratory housing increases morbidity and mortality in research rodents: results of a meta-analysis. BMC Biol. 20, 15 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  177. Tao, Y. et al. Autologous transplant therapy alleviates motor and depressive behaviors in parkinsonian monkeys. Nat. Med. 27, 632–639 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  178. Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. In Proc. IEEE International Conference on Computer Vision 618–626 (IEEE, 2017).

  179. Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. In Proc. Advances in Neural Information Processing Systems 30, 4768–4777 (Curran Associates, 2017).

  180. Cuthbert, B. N. The RDoC framework: facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology. World Psychiatry 13, 28–35 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  181. Anderson, D. J. & Adolphs, R. A framework for studying emotions across species. Cell 157, 187–200 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  182. Zych, A. D. & Gogolla, N. Expressions of emotions across species. Curr. Opin. Neurobiol. 68, 57–66 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  183. Gutierrez-Barragan, D., Ramirez, J. S., Panzeri, S., Xu, T. & Gozzi, A. Evolutionarily conserved fMRI network dynamics in the mouse, macaque and human brain. Nat. Commun. 15, 8518 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  184. Pagani, M., Gutierrez-Barragan, D., de Guzman, A. E., Xu, T. & Gozzi, A. Mapping and comparing fMRI connectivity networks across species. Commun. Biol. 6, 1238 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  185. Wang, X. et al. U-net model for brain extraction: Trained on humans for transfer to non-human primates. Neuroimage 235, 118001 (2021).

    Article  PubMed  Google Scholar 

  186. Zhong, T. et al. nBEST: deep-learning-based non-human primates Brain Extraction and Segmentation Toolbox across ages, sites and species. NeuroImage 295, 120652 (2024).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The manuscript development was supported by a NIH NEW Brain Aging Center Grant (U24AG072701 to F.V.-L. and K.H.W.), the University of Rochester Del Monte Institute for Neuroscience (K.H.W.) and the Research Council of Finland Academy Professor project EmotionAI (grants 336116, 345122 and 359854 to G.Z.).

Author information

Author notes

  1. These authors contributed equally: Adam Turnbull, Ehsan Adeli, Guoying Zhao.

Authors and Affiliations

  1. Department of Psychiatry and Behavioral Sciences, Stanford University, Palo Alto, CA, USA

    Yang Merik Liu, Adam Turnbull, Ehsan Adeli & Feng Vankee-Lin

  2. Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, North Ostrobothnia, Finland

    Guoying Zhao

  3. Department of Neuroscience, University of Rochester Medical Center, Rochester, NY, USA

    Kuan Hong Wang

  4. Palo Alto Health Care System, US Department of Veterans Affairs, Palo Alto, CA, USA

    Feng Vankee-Lin

Authors
  1. Yang Merik Liu
    View author publications

    Search author on:PubMed Google Scholar

  2. Adam Turnbull
    View author publications

    Search author on:PubMed Google Scholar

  3. Ehsan Adeli
    View author publications

    Search author on:PubMed Google Scholar

  4. Guoying Zhao
    View author publications

    Search author on:PubMed Google Scholar

  5. Kuan Hong Wang
    View author publications

    Search author on:PubMed Google Scholar

  6. Feng Vankee-Lin
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Y.M.L. led all aspects of the manuscript, including conceptualization, literature reviewing, framework development, drafting and integration of content. A.T., E.A. and G.Z. contributed to revising the paper and provided expertise in neuroscience theory, AI technology and human behavioral analysis, respectively. K.H.W. co-supervised the project, polished the manuscript, and contributed knowledge on animal behavior and underlying mechanisms. F.V.-L. supervised the project, refined the manuscript, and provided expertise on human behavior and underlying mechanisms.

Corresponding authors

Correspondence to Kuan Hong Wang or Feng Vankee-Lin.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Mental Health thanks Atsushi Noritake and the other, anonymous, reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y.M., Turnbull, A., Adeli, E. et al. A multi-dimensional transfer learning framework for studying reward-guided behaviors across species. Nat. Mental Health 4, 15–29 (2026). https://doi.org/10.1038/s44220-025-00547-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s44220-025-00547-8

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing