Abstract
Influential neurocomputational models emphasize dopamine (DA) as an electrophysiological and neurochemical correlate of reinforcement learning. However, evidence of a specific causal role of DA receptors in learning has been less forthcoming, especially in humans. Here we combine, in a between-subjects design, administration of a high dose of the selective DA D2/3-receptor antagonist sulpiride with genetic analysis of the DA D2 receptor in a behavioral study of reinforcement learning in a sample of 78 healthy male volunteers. In contrast to predictions of prevailing models emphasizing DA’s pivotal role in learning via prediction errors, we found that sulpiride did not disrupt learning, but rather induced profound impairments in choice performance. The disruption was selective for stimuli indicating reward, whereas loss avoidance performance was unaffected. Effects were driven by volunteers with higher serum levels of the drug, and in those with genetically determined lower density of striatal DA D2 receptors. This is the clearest demonstration to date for a causal modulatory role of the DA D2 receptor in choice performance that might be distinct from learning. Our findings challenge current reward prediction error models of reinforcement learning, and suggest that classical animal models emphasizing a role of postsynaptic DA D2 receptors in motivational aspects of reinforcement learning may apply to humans as well.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Ahlenius S, Engel J, Zoller M (1977). Effects of apomorphine and haloperidol on exploratory-behavior and latent learning in mice. Phys Psychol 5: 290–294.
Bai JS, Perron P (1998). Estimating and testing linear models with multiple structural changes. Econometrica 66: 47–78.
Bardgett ME, Depenbrock M, Downs N, Points M, Green L (2009). Dopamine modulates effort-based decision making in rats. Behav Neurosci 123: 242–251.
Bayer HM, Glimcher PW (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47: 129–141.
Beninger RJ, Phillips AG (1981). The effects of pimozide during pairing on the transfer of classical conditioning to an operant discrimination. Pharmacol Biochem Behav 14: 101–105.
Berridge KC, Robinson TE (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28: 309–369.
Blackburn JR, Phillips AG, Fibiger HC (1987). Dopamine and preparatory behavior: I. effects of pimozide. Behav Neurosci 101: 352–360.
Bond A, Lader M (1974). Use of analog scales in rating subjective feelings. Br J Med Psychol 47: 211–218.
Bressan RA, Erlandsson K, Jones HM, Mulligan R, Flanagan RJ, Ell PJ et al (2003). Is regionally selective D2/D3 dopamine occupancy sufficient for atypical antipsychotic effect? An in vivo quantitative [123I]epidepride SPET study of amisulpride-treated patients. Am J Psychiatry 160: 1413–1420.
Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106: 4894–4899.
Calabresi P, Centonze D, Gubellini P, Marfia GA, Pisani A, Sancesario G et al (2000). Synaptic transmission in the striatum: from plasticity to neurodegeneration. Prog Neurobiol 61: 231–265.
Chowdhury R, Guitart-Masip M, Lambert C, Dayan P, Huys Q, Duzel E et al (2013). Dopamine restores reward prediction errors in old age. Nat Neurosci 16: 648–653.
Cohen MX, Krohn-Grimberghe A, Elger CE, Weber B (2007). Dopamine gene predicts the brain's response to dopaminergic drug. Eur J Neurosci 26: 3652–3660.
Daw ND, Kakade S, Dayan P (2002). Opponent interactions between serotonin and dopamine. Neural Netw 15: 603–616.
Dodds CM, Clark L, Dove A, Regenthal R, Baumann F, Bullmore E et al (2009). The dopamine D2 receptor antagonist sulpiride modulates striatal BOLD signal during the manipulation of information in working memory. Psychopharmacology (Berl) 207: 35–45.
Dreyer JK, Herrik KF, Berg RW, Hounsgaard JD (2010). Influence of phasic and tonic dopamine release on receptor activation. J Neurosci 30: 14273–14283.
Eisenegger C, Knoch D, Ebstein RP, Gianotti LRR, Sandor PS, Fehr E (2010). Dopamine receptor D4 polymorphism predicts the effect of L-DOPA on gambling behavior. Biol Psychiatry 67: 702–706.
Eisenegger C, Pedroni A, Rieskamp J, Zehnder C, Ebstein R, Fehr E et al (2013). DAT1 polymorphism determines L-DOPA effects on learning about others' prosociality. PLoS One 8: e67820.
Frank MJ (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17: 51–72.
Frank MJ, Fossella JA (2008). Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology 36: 133–152.
Frank MJ, Seeberger LC, O'Reilly RC (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306: 1940–1943.
Glimcher PW (2011). Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci USA 108 (Suppl 3): 15647–15654.
Goto Y, Grace AA (2005). Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat Neurosci 8: 805–812.
Ikemoto S, Panksepp J (1999). The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev 31: 6–41.
Jocham G, Klein TA, Neumann J, von Cramon DY, Reuter M, Ullsperger M (2009). Dopamine DRD2 polymorphism alters reversal learning and associated neural activity. J Neurosci 29: 3695–3704.
Jocham G, Klein TA, Ullsperger M (2011). Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J Neurosci 31: 1606–1613.
Jonsson EG, Nothen MM, Grunhage F, Farde L, Nakashima Y, Propping P et al (1999). Polymorphisms in the dopamine D2 receptor gene and their relationships to striatal dopamine receptor density of healthy volunteers. Mol Psychiatry 4: 290–296.
Judd CM, McClelland GH (1989) Data Analysis, A Model-Comparison Approach. Orlando, FL, USA.
Kapur S (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiat 160: 13–23.
Kirsch P, Reuter M, Mier D, Vaitl D, Hennig J (2005). Imaging gene-substance interactions: the effect of the DRD2 Taq 1A polymorphism and the dopamine agonist bromocriptine on the brain activation during the anticipation of reward. J Psychophysiol 19: 126–126.
Klein TA, Neumann J, Reuter M, Hennig J, von Cramon DY, Ullsperger M (2007). Genetically determined differences in learning from errors. Science 318: 1642–1645.
Lammel S, Ion DI, Roeper J, Malenka RC (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70: 855–862.
Lex B, Hauber W (2010). The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning. Neurobiol Learn Mem 93: 283–290.
Lovinger DM (2012). Neurotransmitter roles in synaptic modulation, plasticity and learning in the dorsal striatum. Neuropharmacology 58: 951–961.
Matsumoto M, Hikosaka O (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459: 837–U834.
McCabe C, Huber A, Harmer CJ, Cowen PJ (2011). The D2 antagonist sulpiride modulates the neural processing of both rewarding and aversive stimuli in healthy volunteers. Psychopharmacology 217: 271–278.
Mehta MA, Hinton EC, Montgomery AJ, Bantick RA, Grasby PM (2005). Sulpiride and mnemonic function: effects of a dopamine D2 receptor antagonist on working memory, emotional memory and long-term memory in healthy volunteers. J psychopharmacol 19: 29–38.
Mehta MA, McGowan SW, Lawrence AD, Aitken MR, Montgomery AJ, Grasby PM (2003). Systemic sulpiride modulates striatal blood flow: relationships to spatial working memory and planning. Neuroimage 20: 1982–1994.
Mehta MA, Montgomery AJ, Kitamura Y, Grasby PM (2008). Dopamine D2 receptor occupancy levels of acute sulpiride challenges that produce working memory and learning impairments in healthy volunteers. Psychopharmacology 196: 157–165.
Mirenowicz J, Schultz W (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379: 449–451.
Montague PR, Dayan P, Sejnowski TJ (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947.
Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006). Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9: 1057–1063.
Niv Y (2007). Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation? Ann N Y Acad Sci 1104: 357–376.
Palminteri S, Lebreton Ml, Worbe Y, Grabli D, Hartmann A, Pessiglione M (2009). Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes. Proc Natl Acad Sci USA 106: 19179–19184.
Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: 1042–1045.
Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL et al (2008). Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology (Berl) 196: 221–232.
Pohjalainen T, Rinne JO, Nagren K, Lehikoinen P, Anttila K, Syvalahti EK et al (1998). The A1 allele of the human D2 dopamine receptor gene predicts low D2 receptor availability in healthy volunteers. Mol Psychiatry 3: 256–260.
Riba J, Kramer UM, Heldmann M, Richter S, Munte TF (2008). Dopamine agonist increases risk taking but blunts reward-related brain activity. PLoS One 3: e2479.
Rice ME, Cragg SJ (2008). Dopamine spillover after quantal release: rethinking dopamine transmission in the nigrostriatal pathway. Brain Res Rev 58: 303–313.
Ritchie T, Noble EP (1996). [3H]naloxone binding in the human brain: alcoholism and the TaqI A D2 dopamine receptor polymorphism. Brain Res 718: 193–197.
Ritchie T, Noble EP (2003). Association of seven polymorphisms of the D2 dopamine receptor gene with brain receptor-binding characteristics. Neurochem Res 28: 73–82.
Robbins TW, Everitt BJ (1992). Functions of dopamine in the dorsal and ventral striatum. Semin Neurosci 4: 119–127.
Rush CR, Stoops WW, Hays LR, Glaser PEA, Hays LS (2003). Risperidone attenuates the discriminative-stimulus effects of d-amphetamine in humans. J Pharmacol Exp Ther 306: 195–204.
Salamone JD (1994). The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 61: 117–133.
Schultz W (1998). Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1–27.
Schultz W (2002). Getting formal with dopamine and reward. Neuron 36: 241–263.
Schultz W, Dayan P, Montague PR (1997). A neural substrate of prediction and reward. Science 275: 1593–1599.
Seamans JK, Durstewitz D, Christie BR, Stevens CF, Sejnowski TJ (2001). Dopamine D1/D5 receptor modulation of excitatory synaptic inputs to layer V prefrontal cortex neurons. Proc Natl Acad Sci USA 98: 301–306.
Shiner T, Seymour B, Wunderlich K, Hill C, Bhatia KP, Dayan P et al (2012). Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease. Brain 135: 1871–1883.
Smittenaar P, Chase HW, Aarts E, Nusselein B, Bloem BR, Cools R (2012). Decomposing effects of dopaminergic medication in Parkinson's disease on probabilistic action selection–learning or performance? Eur J Neurosci 35: 1144–1151.
Sutton RS, Barto AG (1998). Reinforcement learning: an introduction. IEEE Trans Neural Netw 9: 1054–1054.
Takano A, Suhara T, Yasuno F, Suzuki K, Takahashi H, Morimoto T et al (2006). The antipsychotic sultopride is overdosed—a PET study of drug-induced receptor occupancy in comparison with sulpiride. Int J Neuropsychopharmacol 9: 539–545.
Thompson J, Thomas N, Singleton A, Piggott M, Lloyd S, Perry EK et al (1997). D2 dopamine receptor gene (DRD2) Taq1 A polymorphism: reduced dopamine D2 receptor binding in the human striatum associated with the A1 allele. Pharmacogenetics 7: 479–484.
Treadway MT, Zald DH (2013). Parsing anhedonia: translational models of reward-processing deficits in psychopathology. Curr Dir Psychol Sci 22: 244–249.
van der Schaaf ME, van Schouwenburg MR, Geurts DE, Schellekens AF, Buitelaar JK, Verkes RJ et al (2012). Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning. Cereb Cortex 24: 633–642.
Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ et al (2010). Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron 65: 135–142.
Wang J, O'Donnell P (2001). D(1) dopamine receptors potentiate nmda-mediated excitability increase in layer V prefrontal cortical pyramidal neurons. Cereb Cortex 11: 452–462.
Wise RA (2004). Dopamine, learning and motivation. Nat Rev Neurosci 5: 483–494.
Wise RA, Rompre PP (1989). Brain dopamine and reward. Annu Rev Psychol 40: 191–225.
Xiberas X, Martinot JL, Mallet L, Artiges E, Canal M, Loc'h C et al (2001). In vivo extrastriatal and striatal D2 dopamine receptor blockade by amisulpride in schizophrenia. J Clin Psychopharmacol 21: 207–214.
Zweifel LS, Parker JG, Lobb CJ, Rainwater A, Wall VZ, Fadok JP et al (2009). Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc Natl Acad Sci USA 106: 7281–7288.
Acknowledgements
We gratefully acknowledge the participation of all NIHR Cambridge BioResource (CBR) volunteers. We thank the Cambridge BioResource staff for their help with volunteer recruitment. We also thank members of the Cambridge BioResource SAB and Management Committee for their support given to our study and the National Institute for Health Research Cambridge Biomedical Research Centre for funding. Access to CBR volunteers and their data and samples is governed by the CBR SAB. Documents describing access arrangements and contact details are available at http://www.cambridgebioresource.org.uk. We thank Violetta Dalla and Rudolf Cardinal for their support in the statistical analysis.
Author information
Authors and Affiliations
Corresponding author
Additional information
Author Contributions
CE, MN, UM, LC and TWR designed the research; CE, MN, AL, and UM performed the research; UM and PKG provided medical cover; MN analyzed the data; CE, MN and TWR wrote the paper.
Supplementary Information accompanies the paper on the Neuropsychopharmacology website
Supplementary information
Rights and permissions
About this article
Cite this article
Eisenegger, C., Naef, M., Linssen, A. et al. Role of Dopamine D2 Receptors in Human Reinforcement Learning. Neuropsychopharmacol 39, 2366–2375 (2014). https://doi.org/10.1038/npp.2014.84
Received:
Revised:
Accepted:
Published:
Issue date:
DOI: https://doi.org/10.1038/npp.2014.84
This article is cited by
-
Dopamine regulates decision thresholds in human reinforcement learning in males
Nature Communications (2023)
-
Methylphenidate undermines or enhances divergent creativity depending on baseline dopamine synthesis capacity
Neuropsychopharmacology (2023)
-
Testosterone eliminates strategic prosocial behavior through impacting choice consistency in healthy males
Neuropsychopharmacology (2023)
-
Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients
BMC Psychiatry (2022)
-
Impulsivity and risk-seeking as Bayesian inference under dopaminergic control
Neuropsychopharmacology (2022)
