Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Midbrain dopamine neurons encode decisions for future action

Nature Neuroscience volume 9, pages 1057–1063 (2006)Cite this article

Abstract

Current models of the basal ganglia and dopamine neurons emphasize their role in reinforcement learning. However, the role of dopamine neurons in decision making is still unclear. We recorded from dopamine neurons in monkeys engaged in two types of trial: reference trials in an instructed-choice task and decision trials in a two-armed bandit decision task. We show that the activity of dopamine neurons in the decision setting is modulated according to the value of the upcoming action. Moreover, analysis of the probability matching strategy in the decision trials revealed that the dopamine population activity and not the reward during reference trials determines choice behavior. Because dopamine neurons do not have spatial or motor properties, we conclude that immediate decisions are likely to be generated elsewhere and conveyed to the dopamine neurons, which play a role in shaping long-term decision policy through dynamic modulation of the efficacy of basal ganglia synapses.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Behavioral tasks and response parameters.
Figure 2: Contribution to probability matching behavior.
Figure 3: Eye positions in reference and decision trials.
Figure 4: Dopamine neurons code the TD error of action value.
Figure 5: Time course of the responses of dopamine neurons in the reference and decision tasks.

Similar content being viewed by others

References

  1. Herrnstein, R.J. On the law of effect. J. Exp. Anal. Behav. 13, 243–266 (1970).

    Article  CAS  Google Scholar 

  2. Tversky, A. & Kahneman, D. The framing of decisions and the psychology of choice. Science 211, 453–458 (1981).

    Article  CAS  Google Scholar 

  3. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  4. Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).

    Article  CAS  Google Scholar 

  5. Romo, R., Hernandez, A. & Zainos, A. Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41, 165–173 (2004).

    Article  CAS  Google Scholar 

  6. Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).

    Article  CAS  Google Scholar 

  7. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

    Article  CAS  Google Scholar 

  8. Pagnoni, G., Zink, C.F., Montague, P.R. & Berns, G.S. Activity in human ventral striatum locked to errors of reward prediction. Nat. Neurosci. 5, 97–98 (2002).

    Article  CAS  Google Scholar 

  9. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  10. Rummery, G.A. & Niranjan, M. On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166 (Engineering Department, Cambridge University, Cambridge, UK, 1994).

    Google Scholar 

  11. Watkins, C.J.C.H. & Dayan, P. Q learning. Mach. Learn. 8, 279–292 (1992).

    Google Scholar 

  12. Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).

    Article  CAS  Google Scholar 

  13. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  14. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

    Article  CAS  Google Scholar 

  15. Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913–9923 (2003).

    Article  CAS  Google Scholar 

  16. Morris, G., Arkadir, D., Nevet, A., Vaadia, E. & Bergman, H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133–143 (2004).

    Article  CAS  Google Scholar 

  17. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).

    Article  CAS  Google Scholar 

  18. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

    Article  CAS  Google Scholar 

  19. Gurney, K., Prescott, T.J., Wickens, J.R. & Redgrave, P. Computational models of the basal ganglia: from robots to membranes. Trends Neurosci. 27, 453–459 (2004).

    Article  CAS  Google Scholar 

  20. Mink, J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996).

    Article  CAS  Google Scholar 

  21. Bar-Gad, I., Morris, G. & Bergman, H. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71, 439–473 (2003).

    Article  Google Scholar 

  22. Barto, A.G. Adaptive critics and the basal ganglia. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995).

    Google Scholar 

  23. Reynolds, J.N., Hyland, B.I. & Wickens, J.R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001).

    Article  CAS  Google Scholar 

  24. McClure, S.M., Daw, N.D. & Montague, P.R. A computational substrate for incentive salience. Trends Neurosci. 26, 423–428 (2003).

    Article  CAS  Google Scholar 

  25. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).

    Article  CAS  Google Scholar 

  26. Wolford, G., Miller, M.B. & Gazzaniga, M. The left hemisphere's role in hypothesis formation. J. Neurosci. 20, RC64 (2000).

    Article  CAS  Google Scholar 

  27. Vulkan, N. An economist's perspective on probability matching. J. Econ. Surv. 14, 101–118 (2000).

    Article  Google Scholar 

  28. Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005).

    Article  Google Scholar 

  29. Dommett, E. et al. How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476–1479 (2005).

    Article  CAS  Google Scholar 

  30. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).

    Article  CAS  Google Scholar 

  31. Cragg, S.J. & Rice, M.E. DAncing past the DAT at a DA synapse. Trends Neurosci. 27, 270–277 (2004).

    Article  CAS  Google Scholar 

  32. Wickens, J.R. & Arbuthnot, G.W. Structural and functional interactions in the striatum at the receptor level. in Dopamine (eds. Dunnett, S.B., Bentivoglio, M., Bjorklund, A. & Hokfelt, T.) 199–236 (Elsevier, Amsterdam, 2005).

    Chapter  Google Scholar 

  33. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).

    Article  CAS  Google Scholar 

  34. Szabo, J. & Cowan, W.M. A stereotaxic atlas of the brain of the cynomolgus monkey (Macaca fascicularis). J. Comp. Neurol. 222, 265–300 (1984).

    Article  CAS  Google Scholar 

  35. Martin, R.F. & Bowden, D.M. Primate Brain Maps: Structure of the Macaque Brain (Elsevier Science, Amsterdam, 2000).

    Google Scholar 

  36. Grace, A.A. & Bunney, B.S. Intracellular and extracellular electrophysiology of nigral dopaminergic neurons–1. Identification and characterization. Neuroscience 10, 301–315 (1983).

    Article  CAS  Google Scholar 

  37. Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042 (2004).

    Article  CAS  Google Scholar 

  38. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

    Article  CAS  Google Scholar 

  39. Nevet, A., Morris, G., Saban, G., Fainstein, N. & Bergman, H. Discharge rate of substantia nigra pars reticulata neurons is reduced in non-parkinsonian monkeys with apomorphine-induced orofacial dyskinesia. J. Neurophysiol. 92, 1973–1981 (2004).

    Article  Google Scholar 

  40. Sokal, R.R. & Rohlf, F.J. Biometry (W.H. Freeman & Co., New York, 1981).

    Google Scholar 

Download references

Acknowledgements

We thank Y. Prut, Y. Engel, Y. Ritov, N. Daw, Y. Niv and R. Paz for fruitful discussions and comments on earlier versions of this manuscript; and G. Goelman and V. Sharkansky for technical assistance. This study was partly supported by a Center of Excellence grant administered by the Israel Science Foundation (ISF) and by a 'Fighting against Parkinson' grant administered by the Netherlands Friends of the Hebrew University (HUNA). G.M. was supported by a Horowitz fellowship.

Author information

Authors and Affiliations

  1. Interdisciplinary Center for Neural Computation (ICNC), Hebrew University, Jerusalem, Israel

    Genela Morris, Eilon Vaadia & Hagai Bergman

  2. Department of Physiology, Hebrew University, PO Box 12272, Jerusalem, 91220, Israel

    Genela Morris, Alon Nevet, David Arkadir, Eilon Vaadia & Hagai Bergman

Authors
  1. Genela Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alon Nevet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Arkadir
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eilon Vaadia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hagai Bergman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Genela Morris.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Choice triggered averaging (CTA) functions in decision trials. (PDF 321 kb)

Supplementary Fig. 2

Conditioning stimulus responses according to choice value. (PDF 168 kb)

Supplementary Note (PDF 196 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Morris, G., Nevet, A., Arkadir, D. et al. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9, 1057–1063 (2006). https://doi.org/10.1038/nn1743

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1743

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing