Abstract
Psychosis is linked to dysregulation of the neuromodulator dopamine and antipsychotic drugs (APDs) work by blocking dopamine receptors. Dopamine-modulated disruption of latent inhibition (LI) and conditioned avoidance response (CAR) have served as standard animal models of psychosis and antipsychotic action, respectively. Meanwhile, the ‘temporal difference’ algorithm (TD) has emerged as the leading computational model of dopamine neuron firing. In this report TD is extended to include action at the level of dopamine receptors in order to explain a number of behavioral phenomena including the dose-dependent disruption of CAR by APDs, the temporal dissociation of the effects of APDs on receptors vs behavior, the facilitation of LI by APDs, and the disruption of LI by amphetamine. The model also predicts an APD-induced change to the latency profile of CAR—a novel prediction that is verified experimentally. The model's primary contribution is to link dopamine neuron firing, receptor manipulation, and behavior within a common formal framework that may offer insights into clinical observations.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Abi-Dargham A (2004). Do we still believe in the dopamine hypothesis? New data bring new evidence. Int J Neuropsychopharmacol 7 (Suppl 1): S1–S5.
Agid O, Kapur S, Arenovich T, Zipursky RB (2003). Delayed-onset hypothesis of antipsychotic action: a hypothesis tested and rejected. Arch Gen Psychiatry 60: 1228–1234.
Anisman H, Irwin J, Zacharko RM, Tombaugh TN (1982). Effects of dopamine receptor blockade on avoidance performance: assessment of effects on cue-shock and response-outcome associations. Behav Neural Biol 36: 280–290.
Baruch I, Hemsley DR, Gray JA (1988). Differential performance of acute and chronic schizophrenics in a latent inhibition task. J Nerv Ment Dis 176: 598–606.
Bayer HM, Glimcher PW (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47: 129–141.
Beninger RJ (1989a). The role of serotonin and dopamine in learning to avoid aversive stimuli. In: Archer T, Nilsson L (eds). Aversion, Avoidance, and Anxiety: Perspective on Aversively Motivated Behavior. Lawrence Erlbaum Associates: Hillsdale, New Jersey. pp 265–284.
Beninger RJ (1989b). Dissociating the effects of altered dopaminergic function on performance and learning. Brain Res Bull 23: 365–371.
Berridge KC, Robinson TE (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28: 309–369.
Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002). Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal coretx. Neurosci Biobehav Rev 26: 321–352.
Connell PH (1958). Amphetamine Psychosis. Chapman and Hall: London.
Courvoisier S (1956). Pharmacodynamic basis for the use of chlorpromazine in psychiatry. Q Rev Psychiatry Neurol 17: 25–37.
Dayan P, Balleine BW (2002). Reward, motivation and reinforcement learning. Neuron 36: 285–298.
Feldon J, Shofel A, Weiner I (1991). Latent inhibition is unaffected by direct dopamine agonists. Pharmacol Biochem Behav 38: 309–314.
Fibiger HC, Zis AP, Phillips AG (1975). Haloperidol-induced disruption of conditioned avoidance responding: attenuation by prior training or by anticholinergic drugs. Eur J Pharmacol 30: 309–314.
Garris PA, Christensen JR, Rebec GV, Wightman RM (1997). Real-time measurement of electrically evoked extracellular dopamine in the striatum of freely moving rats. J Neurochem 68: 152–161.
Garris PA, Kilpatrick M, Bunin MA, Michael D, Walker QD, Wightman RM (1999). Dissociation of dopamine release in the nucleus accumbens from striatal self-stimulation. Nature 398: 67–69.
Grace AA (1995). The tonic/phasic model of dopamine system regulation: its relevance for understanding how stimulant abuse can alter basal ganglia function. Drug Alcohol Depend 37: 111–129.
Grace AA (1991). Phasic vs tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41: 1–24.
Grace AA, Bunney BS, Moore H, Todd CL (1997). Dopamine-cell depolarization block as a model for the therapeutic actions of antipsychotic drugs. Trends Neurosci 20: 31–37.
Gray NS, Pickering AD, Hemsley DR, Dawling S, Gray JA (1992). Abolition of latent inhibition by a single 5 mg dose of d-amphetamine in man. Psychopharmacology (Berlin) 107: 425–430.
Hollerman J, Schultz W (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1: 304–309.
Horvitz JC (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96: 651–656.
Janssen PAJ, Niemegeers CJE, Schellekens KHL (1965). Is it possible to predict the clinical effects of neuroleptic drugs (major tranquilizers) from animal data? Arzneimittelforschung 15: 104–117.
Joseph MH, Datla K, Young AMJ (2003). The interpretation of the measurement of nucleus accumbens dopamine by vivo dialysis: the kick, the craving or the cognition. Neurosci Biobehav Rev 27: 527–541.
Joseph MH, Peters SL, Moran PM, Grigorvan GA, Young AM, Gray JA (2000). Modulation of latent inhibition in the rat by altered dopamine transmission in the nucleus accumbens at the time of conditioning. Neuroscience 101: 921–930.
Kapur S (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology and pharmacology in Schizophrenia. Am J Psychiatry 160: 13–23.
Kapur S, Mamo D (2003). Half a century of antipsychotics and still a central role for dopamine D2 receptors. Prog Neuropsychopharmacol Biol Psychiatry 27: 1081–1090.
Kapur S, VanderSpek SC, Brownlee BA, Nobrega JN (2003). Antipsychotic dosing in preclinical models is often unrepresentative of the clinical condition: a suggested solution based on in vivo occupancy. J Pharmacol Exp Ther 305: 625–631.
Killcross AS, Dickinson A, Robbins TW (1994). Amphetamine-induced disruptions of latent inhibition are reinforcer mediated: implications for animal models of schizophrenic attentional dysfunction. Psychopharmacology 115: 185–195.
Li M, Parkes J, Fletcher PJ, Kapur S (2004). Evaluation of the motor initiation hypothesis of APD-induced conditioned avoidance decreases. Pharmacol Biochem Behav 78: 811–819.
Lubow RE (1989). Latent Inhibition and Conditioned Attention Theory. Cambridge University Press: Cambridge.
Lubow RE (1997). Latent inhibition as a measure of learned inattention: some problems and solutions. Behav Brain Res 88: 75–83.
Lubow RE (2005). Construct validity of the animal latent inhibition model of selective attention deficits in schizophrenia. Schizophr Bull 31: 139–153.
Lubow RE, Moore AU (1959). Latent inhibition: the effect of nonreinforced pre-exposure to the conditional stimulus. J Comp Physiol Psychol 52: 415–419.
Mackintosh NJ (1974). The Psychology of Animal Learning. Academic Press: New York.
Mackintosh NJ (1975). A theory of attention: variations in the associability of stimuli with reinforcement. Psychol Rev 82: 276–298.
McClure SM, Daw N, Montague PR (2003). A computational substrate for incentive salience. Trends Neurosci 26: 423–428.
Mirenowicz J, Schultz W (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379: 449–451.
Montague PR, Dayan P, Person C, Sejnowski TJ (1995). Bee foraging in uncertain environments using predictive Hebbian learning. Nature 377: 725–728.
Montague PR, Dayan P, Sejnowski TJ (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947.
Moser PC, Hitchcock JM, Lister S, Moran PM (2000). The pharmacology of latent inhibition as an animal model of schizophrenia. Brain Res Rev 33: 275–307.
Phillips PEM, Stuber GD, Helen MLAV, Wightman RM, Carelli RM (2003). Subsecond dopamine release promotes cocaine seeking. Nature 422: 614–618.
Rebec GV, Christensen JR, Guerra C, Bardo MT (1997). Regional and temporal differences in real-time dopamine efflux in the nucleus accumbens during free-chioce novelty. Brain Res 776: 61–67.
Redgrave P, Prescott TJ, Gurney K (1999). Is the short-latency dopamine response too short to signal reward error? Trends in Neurosci 22: 146–151.
Redish DA (2004). Addiction as a computational process gone awry. Science 306: 1944–1947.
Robbins TW, Everitt BJ (1996). Neurobehavioural mechanisms of reward and motivation. Curr Opin Neurobiol 6: 228–236.
Robinson DL, Venton BJ, Heien MLAV, Wightman RM (2003). Detecting subsecond dopamine release with fast-scan cyclic voltammetry in vivo. Clin Chem 49: 1763–1773.
Roitman MF, Stuber GD, Phillips PEM, Wightman RM, Carelli RM (2004). Dopamine operates as a subsecond modulator of food seeking. J Neurosci 24: 1265–1271.
Romo R, Schultz W (1990). Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63: 592–606.
Salamone JD, Cousins MS, Bucher S (1994). Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav Brain Res 65: 221–229.
Schultz W (1998). Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1–27.
Schultz W, Apicella P, Ljungberg T (1992). Response of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13: 900–913.
Schultz W, Dayan P, Montague PR (1997). A neural substrate of prediction and reward. Science 275: 1593–1599.
Seamans JK, Yang CR (2004). The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol 74: 1–57.
Segal DS, Kuczenski R (1992). In vivo microdialysis reveals a diminished amphetamine-induced DA response corresponding to behavioral sensitization produced by repeated amphetamine pretreatment. Brain Res 571: 330–337.
Solomon PR, Crider A, Winkelman JW, Turi A, Kamer RM, Kaplan LJ (1981). Disrupted latent inhibition in the rat with chronic amphetamine or haloperidol-induced supersensitivity: relationship to schizophrenic attention disorder. Biol Psychiatry 16: 519–537.
Tesauro GJ (1994). TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation 6: 215–219.
Wadenberg MG, Soliman A, Vanderspek SC, Kapur S (2001). Dopamine D2 receptor occupancy is a common mechanism underlying animal models of antipsychotics and their clinical effects. Neuropsychopharmacology 25: 633–641.
Weiner I (1990). Neural substrates of latent inhibition: the switching model. Psychol Bull 108: 442–461.
Weiner I (2003). The ‘two-headed’ latent inhibition model of schizophrenia: modeling positive and negative symptoms and their treatment. Psychopharmacology (Berlin) 169: 257–297.
Weiner I, Feldon J, Katz Y (1987). Facilitation of the expression but not the acquisition of latent inhibition by haloperidol in rats. Pharmacol, Biochem Behav 26: 241–246.
Weiner I, Lubow RE, Feldon J (1981). Chronic amphetamine and latent inhibition. Behav Brain Res 2: 285–286.
Weiner I, Lubow RE, Feldon J (1988). Disruption of latent inhibition by acute administration of low doses of amphetamine. Pharmacol Biochem Behav 30: 871–878.
Wise RA (1982). Neuroleptics and operant behavior: the anhedonia hypothesis. Behav Brain Sci 5: 39–87.
Author information
Authors and Affiliations
Corresponding author
Additional information
Supplementary Information accompanies the paper on the Neuropsychopharmacology website (http://www.nature.com/npp).
Supplementary information
APPENDIX
APPENDIX
SUPPLEMENTARY MATERIAL
All procedures were approved by the animal care committee at the Centre for Addiction and Mental Health, and adhered to the guidelines established by the Canadian Council on Animal Care.
TD Simulation of Conditioned Avoidance
In TD, the Value of each state is intended as an estimate of all future reward following that state:
where r(t) is the reward associated with the state of time t, and r(t+1) is the reward associated with the following state etc. γ (between 0 and 1) is the discount factor that controls the degree to which future rewards are discounted over immediate rewards (eg a dollar tomorrow is worth slightly less than a dollar today). γ>0.9 is commonly used, and we arbitrarily select γ=0.93. As with the number of interval states, we select a value that empirically matches the quantitative properties of the observed data, without affecting the qualitative performance of the model.
The Value of each state must be estimated by repeated exposure to that state over multiple trials. To avoid the need to wait indefinitely for all future rewards before updating V(t), TD makes use of the recursive definition:
appealing to the ability of V(t+1) to itself estimate r(t+1)+r(t+2)+etc. The ‘temporal difference’ between the left and right hand sides of Equation (7) yields an error term, which is then used to update V(t) towards r(t)+V(t+1) (see Equation (1)). The assumption is that r(t)+V(t+1) is a more accurate estimate of the true future reward because it incorporates real reward from the environment (ie r(t)). These theoretical foundations of TD were formulated long before the model's application to the dopamine system.
In the simulation of conditioned avoidance, four internal states were used, along with a learning rate, α=0.5, and discount factor γ=0.93. The drug free sessions were simulated by setting the ‘dopamine’ parameter, θ=0. Low, medium, and high doses of APD were simulated by setting θ=−0.2, θ=−03, θ=−0.4, respectively. The seven sessions of Figure 2 were modeled as 7 × 30=210 consecutive simulated trials, with only θ being varied from session to session as appropriate.
We note that TD is usually applied to appetitive tasks, with positive prediction error being associated with ‘better than expected’ and negative prediction error with ‘worse than expected’. This presents a problem for modeling aversive paradigms. Although dopamine is clearly implicated in such paradigms, APD-induced dopamine blockade does not appear to render the CS ‘worse’ or more aversive, but apparently less aversive. Our solution involves treating the shock in the same way as rewards have been treated in the past—by setting r(US) to a positive value. The tacit assumption being made is that the dopamine system mediates the learning of all salient events, whether appetitive or aversive. Electrophysiological evidence of dopamine neuron selectivity for appetitive events (Mirenowicz and Schultz, 1996) undermines this assumption, although microdialysis and voltammetry studies suggest that dopamine is also released in response to aversive events (Joseph et al, 2003), and indeed a more general relationship between dopamine and salience has been posited (Horvitz, 2000). The resolution of this debate will have important implications for how aversive paradigms such as CA are modeled using TD in the future.
Latent Inhibition Animal Experiment
The following describes experiment 2 in (Weiner et al, 1988). Seventy-two male Wistar rats served as subjects. They were randomly assigned to one of the eight experimental conditions in a 2 × 2 × 2 design, consisting of stimulus pre-exposure/no pre-exposure, drug/no drug during pre-exposure, and drug/no drug during conditioning. The conditioned stimulus was a 5-s, 2.8 kHz tone and the US was a shock (1.0 mA) supplied to the grid floor. Each animal in the pre-exposed group (PE) was placed in the shuttle box and received 50, 5-s tone presentations without consequence. The non pre-exposed (NPE) animals were confined to the shuttle box for an identical period of time, but were not presented with the tone. Twenty-four hr after pre-exposure, each animal was placed in the shuttle box and received 60 trials of avoidance training. Each avoidance trial started with a 5-s tone followed by a 30-s shock, the tone remaining on with the shock. The total number of avoidance responses (shuttle crossings) was recorded. The appropriate drug, either 1.5 mg/kg dl-amphetamine sulphate dissolved in 1 ml of isotonic saline or an equivalent volume of saline, was administered intraperitoneally 15 min prior to the start of each stage (pre-exposure and conditioning). Subsequent reviews (Moser et al, 2000; Weiner, 1990) suggest that the conditioning phase rather than the pre-exposure phase is the important stage for pharmacological manipulation of LI, and so we focus on the groups that received saline during pre-exposure.
Rights and permissions
About this article
Cite this article
Smith, A., Li, M., Becker, S. et al. Linking Animal Models of Psychosis to Computational Models of Dopamine Function. Neuropsychopharmacol 32, 54–66 (2007). https://doi.org/10.1038/sj.npp.1301086
Received:
Revised:
Accepted:
Published:
Issue date:
DOI: https://doi.org/10.1038/sj.npp.1301086
Keywords
This article is cited by
-
A critical evaluation of dynamical systems models of bipolar disorder
Translational Psychiatry (2022)
-
The Functions of Dopamine in Operant Conditioned Reflexes
Neuroscience and Behavioral Physiology (2019)
-
Regulation of striatal dopamine responsiveness by Notch/RBP-J signaling
Translational Psychiatry (2017)
-
Pro-Cognitive and Antipsychotic Efficacy of the α7 Nicotinic Partial Agonist SSR180711 in Pharmacological and Neurodevelopmental Latent Inhibition Models of Schizophrenia
Neuropsychopharmacology (2009)
