Abstract
Learning that certain actions lead to risky rewards is critical for biological, social, and economic survival, but the precise neural mechanisms of such reward-guided learning remain unclear. Here, we show that the human nucleus accumbens plays a key role in learning about risks by representing reward value. We recorded electrophysiological activity directly from the nucleus accumbens of five patients undergoing deep brain stimulation for treatment of refractory major depression. Patients engaged in a simple reward-learning task in which they first learned stimulus-outcome associations (learning task), and then were able to choose from among the learned stimuli (choosing task). During the learning task, nucleus accumbens activity reflected potential and received reward values both during the cue stimulus and during the feedback. During the choosing task, there was no nucleus accumbens activity during the cue stimulus, but feedback-related activity was pronounced and similar to that during the learning task. This pattern of results is inconsistent with a prediction error response. Finally, analyses of cross-correlations between the accumbens and simultaneous recordings of medial frontal cortex suggest a dynamic interaction between these structures. The high spatial and temporal resolution of these recordings provides novel insights into the timing of activity in the human nucleus accumbens, its functions during reward-guided learning and decision-making, and its interactions with medial frontal cortex.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Abler B, Walter H, Erk S, Kammerer H, Spitzer M (2006). Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage 31: 790–795.
Albertin SV, Mulder AB, Tabuchi E, Zugaro MB, Wiener SI (2000). Lesions of the medial shell of the nucleus accumbens impair rats in finding larger rewards, but spare reward-seeking behavior. Behav Brain Res 117: 173–183.
Atallah HE, Lopez-Paniagua D, Rudy JW, O'Reilly RC (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10: 126–131.
Becerra L, Breiter HC, Wise R, Gonzalez RG, Borsook D (2001). Reward circuitry activation by noxious thermal stimuli. Neuron 32: 927–946.
Berendse HW, Groenewegen HJ (1990). Organization of the thalamostriatal projections in the rat, with special emphasis on the ventral striatum. J Comp Neurol 299: 187–228.
Block AE, Dhanji H, Thompson-Tardif SF, Floresco SB (2007). Thalamic-prefrontal cortical-ventral striatal circuitry mediates dissociable components of strategy set shifting. Cereb Cortex 17: 1625–1636.
Bowman EM, Aigner TG, Richmond BJ (1996). Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75: 1061–1073.
Brady AM, O'Donnell P (2004). Dopaminergic modulation of prefrontal cortical input to nucleus accumbens neurons in vivo. J Neurosci 24: 1040–1049.
Breiter HC, Rosen BR (1999). Functional magnetic resonance imaging of brain reward circuitry in the human. Ann NY Acad Sci 877: 523–547.
Cardinal RN (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw 19: 1277–1301.
Cardinal RN, Howes NJ (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci 6: 37.
Carelli RM, Deadwyler SA (1997). Cellular mechanisms underlying reinforcement-related processing in the nucleus accumbens: electrophysiological studies in behaving animals. Pharmacol Biochem Behav 57: 495–504.
Carr DB, Sesack SR (2000). Projections from the rat prefrontal cortex to the ventral tegmental area: target specificity in the synaptic associations with mesoaccumbens and mesocortical neurons. J Neurosci 20: 3864–3873.
Cohen MX, Heller AS, Ranganath C (2005). Functional connectivity with anterior cingulate and orbitofrontal cortices during decision-making. Brain Res Cogn Brain Res 23: 61–70.
Cohen MX, Ranganath C (2007). Reinforcement learning signals predict future decisions. J Neurosci 27: 371–378.
Cromwell HC, Hassani OK, Schultz W (2005). Relative reward processing in primate striatum. Exp Brain Res 162: 520–525.
Day JJ, Carelli RM (2007). The nucleus accumbens and Pavlovian reward learning. Neuroscientist 13: 148–159.
Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072–3077.
Epstein J, Pan H, Kocsis JH, Yang Y, Butler T, Chusid J et al (2006). Lack of ventral striatal response to positive stimuli in depressed vs normal subjects. Am J Psychiatry 163: 1784–1790.
Ernst M, Nelson EE, Jazbec S, McClure EB, Monk CS, Leibenluft E et al (2005). Amygdala and nucleus accumbens in responses to receipt and omission of gains in adults and adolescents. Neuroimage 25: 1279–1291.
Finch DM (1996). Neurophysiology of converging synaptic inputs from the rat prefrontal cortex, amygdala, midline thalamus, and hippocampal formation onto single neurons of the caudate/putamen and nucleus accumbens. Hippocampus 6: 495–512.
Fiorillo CD, Tobler PN, Schultz W (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902.
Frank MJ, Woroch BS, Curran T (2005). Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47: 495–501.
Gehring WJ, Willoughby AR (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295: 2279–2282.
Groenewegen HJ, Wright CI, Beijer AV, Voorn P (1999). Convergence and segregation of ventral striatal inputs and outputs. Ann NY Acad Sci 877: 49–63.
Haber SN, Fudge JL, McFarland NR (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20: 2369–2382.
Haber SN, Kim KS, Mailly P, Calzavara R (2006). Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26: 8368–8376.
Haber SN, McFarland NR (1999). The concept of the ventral striatum in nonhuman primates. Ann NY Acad Sci 877: 33–48.
Jackson ME, Frost AS, Moghaddam B (2001). Stimulation of prefrontal cortex at physiologically relevant frequencies inhibits dopamine release in the nucleus accumbens. J Neurochem 78: 920–923.
Kasanetz F, Riquelme LA, O'Donnell P, Murer MG (2006). Turning off cortical ensembles stops striatal up states and elicits phase perturbations in cortical and striatal slow oscillations in rat in vivo. J Physiol 577: 97–113.
Knutson B, Adams CM, Fong GW, Hommer D (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21: RC159.
Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH (2008). Neural responses to monetary incentives in major depression. Biol Psychiatry 63: 686–692.
Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002). A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413–417.
Logothetis NK (2002). The neural basis of the blood-oxygen-level-dependent functional magnetic resonance imaging signal. Philos Trans R Soc Lond B Biol Sci 357: 1003–1037.
Maier A, Wilke M, Aura C, Zhu C, Ye FQ, Leopold DA (2008). Divergence of fMRI and neural signals in V1 during perceptual suppression in the awake monkey. Nat Neurosci 11: 1193–1200.
Matthews SC, Simmons AN, Lane SD, Paulus MP (2004). Selective activation of the nucleus accumbens during risk-taking decision making. Neuroreport 15: 2123–2127.
Mayberg HS, Lozano AM, Voon V, McNeely HE, Seminowicz D, Hamani C et al (2005). Deep brain stimulation for treatment-resistant depression. Neuron 45: 651–660.
McClure SM, Berns GS, Montague PR (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339–346.
McCracken CB, Grace AA (2007). High-frequency deep brain stimulation of the nucleus accumbens region suppresses neuronal activity and selectively modulates afferent drive in rat orbitofrontal cortex in vivo. J Neurosci 27: 12601–12610.
McIntyre CC, Savasta M, Kerkerian-Le Goff L, Vitek JL (2004). Uncovering the mechanism(s) of action of deep brain stimulation: activation, inhibition, or both. Clin Neurophysiol 115: 1239–1248.
Mogenson GJ, Swanson LW, Wu M (1983). Neural projections from nucleus accumbens to globus pallidus, substantia innominata, and lateral preoptic-lateral hypothalamic area: an anatomical and electrophysiological investigation in the rat. J Neurosci 3: 189–202.
Morgane PJ, Galler JR, Mokler DJ (2005). A review of systems and networks of the limbic forebrain/limbic midbrain. Prog Neurobiol 75: 143–160.
Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006). Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9: 1057–1063.
Nieuwenhuis S, Holroyd CB, Mol N, Coles MG (2004). Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci Biobehav Rev 28: 441–448.
O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003). Temporal difference models and reward-related learning in the human brain. Neuron 38: 329–337.
O'Doherty JP, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452–454.
Overstreet DH, Friedman E, Mathe AA, Yadid G (2005). The Flinders Sensitive Line rat: a selectively bred putative animal model of depression. Neurosci Biobehav Rev 29: 739–759. E-pub April 2005, 2022.
Pothuizen HH, Jongen-Relo AL, Feldon J, Yee BK (2005). Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. Eur J Neurosci 22: 2605–2616.
Powell EW, Leman RB (1976). Connections of the nucleus accumbens. Brain Res 105: 389–403.
Preuschoff K, Bossaerts P, Quartz SR (2006). Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51: 381–390.
Redgrave P, Gurney K (2006). The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7: 967–975.
Rodriguez PF, Aron AR, Poldrack RA (2006). Ventral-striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Hum Brain Mapp 27: 306–313.
Roitman MF, Wheeler RA, Carelli RM (2005). Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45: 587–597.
Samejima K, Ueda Y, Doya K, Kimura M (2005). Representation of action-specific reward values in the striatum. Science 310: 1337–1340.
Schlaepfer T, Cohen MX, Frick C, Kosel M, Brodesser D, Axmacher N et al (2008). Deep brain stimulation to reward circuitry alleviates anhedonia in refractory major depression. Neuropsychopharmacology 33: 368–377.
Schultz W, Apicella P, Scarnati E, Ljungberg T (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610.
Shidara M, Aigner TG, Richmond BJ (1998). Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18: 2613–2625.
Small DM, Zatorre RJ, Dagher A, Evans AC, Jones-Gotman M (2001). Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124: 1720–1733.
Sturm V, Lenartz D, Koulousakis A, Treuer H, Herholz K, Klein JC et al (2003). The nucleus accumbens: a target for deep brain stimulation in obsessive-compulsive- and anxiety-disorders. J Chem Neuroanat 26: 293–299.
Wolf ME, Mangiavacchi S, Sun X (2003). Mechanisms by which dopamine receptors may influence synaptic plasticity. Ann NY Acad Sci 1003: 241–249.
Yacubian J, Glascher J, Schroeder K, Sommer T, Braus DF, Buchel C (2006). Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26: 9530–9537.
Young AM (2004). Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Methods 138: 57–63.
Acknowledgements
This study was partly funded by Medtronic Inc. and a predoctoral NIDA NRSA to MXC. We thank Caroline Frick, Markus Kosel, and Daniela Rottländer for assistance with the patients, and Michael Frank for useful and insightful discussions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Disclosure/Conflict of Interest
The authors declare no conflict of interest.
Supplementary Information accompanies the paper on the Neuropsychopharmacology website (http://www.nature.com/npp)
Supplementary information
Rights and permissions
About this article
Cite this article
Cohen, M., Axmacher, N., Lenartz, D. et al. Neuroelectric Signatures of Reward Learning and Decision-Making in the Human Nucleus Accumbens. Neuropsychopharmacol 34, 1649–1658 (2009). https://doi.org/10.1038/npp.2008.222
Received:
Revised:
Accepted:
Published:
Issue date:
DOI: https://doi.org/10.1038/npp.2008.222
Keywords
This article is cited by
-
Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases
Nature Communications (2024)
-
Linking connectivity of deep brain stimulation of nucleus accumbens area with clinical depression improvements: a retrospective longitudinal case series
European Archives of Psychiatry and Clinical Neuroscience (2024)
-
Nucleus accumbens deep brain stimulation for a patient with self-injurious behavior and autism spectrum disorder: functional and structural changes of the brain: report of a case and review of literature
Acta Neurochirurgica (2017)
-
Neuromodulation for Treatment Resistant Depression: State of the Art and Recommendations for Clinical and Scientific Conduct
Brain Topography (2014)
-
The Reward Circuit: Linking Primate Anatomy and Human Imaging
Neuropsychopharmacology (2010)
