Elsevier

NeuroImage

Volume 20, Issue 2, October 2003, Pages 1086-1095
NeuroImage

Regular article
Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system:: an event-related fMRI study

https://doi.org/10.1016/S1053-8119(03)00381-1Get rights and content

Abstract

Findings from animal as well as human neuroimaging studies suggest that reward delivery is associated with the activation of subcortical limbic and prefrontal brain regions, including the thalamus, the striatum, the anterior cingulate and the prefrontal cortex. The aim of the present study was to explore if these reward-sensitive regions are also activated during the anticipation of reinforcers that vary with regard to their motivational value. A differential conditioning paradigm was performed, with the presentation of a rewarded reaction time task serving as the unconditioned stimulus (US). Depending on their reaction time, subjects were given (or not given) a monetary reward, or were presented with a verbal feedback consisting of being fast or slow. In a third control condition no task needed to be executed. Each of the three conditions was introduced by a different visual cue (CS). Brain activation of 27 subjects was recorded using event-related functional magnetic resonance imaging. The results showed significant activation of the substantia nigra, thalamic, striatal, and orbitofrontal brain regions as well as of the insula and the anterior cingulate during the presentation of a CS signalling a rewarded task. The anticipation of a monetary reward produced stronger activation in these regions than the anticipation of positive verbal feedback. The results are interpreted as reflecting the motivation-dependent reactivity of the brain reward system with highly motivating stimuli (monetary reward) leading to a stronger activation than those less motivating ones (verbal reward).

Introduction

Brain regions that are responsive to reward delivery have been studied intensely over recent years. Findings from animal studies point to the important role of the striatum, the orbitofrontal cortex Rolls, 1999, Schultz et al., 2000, the thalamus Oyoshi et al., 1996, Komura et al., 2001, and the anterior cingulate Bussey et al., 1997, Parkinson et al., 2000 for the processing of reward stimuli.

In humans comparable structures have been identified by using modern neuroimaging techniques such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET). Here, the application of primary reinforcers (e.g., pleasant taste, touch, or smell) led to an activation of the orbitofrontal and prefrontal cortex, the insula, the anterior cingulate as well as the striatum Francis et al., 1999, O'Doherty et al., 2002. The presentation of secondary reinforcers provoked similar activation patterns. In a study by Thut et al. (1997), the prefrontal, midbrain, and thalamic activation increased when correct responses in a go/nogo task were rewarded with money compared to verbal reinforcement. Delgado et al. (2000) used a gambling task, where subjects could win or lose money. Both conditions provoked activation in the bilateral caudate and the left ventral striatum. In another type of gambling task, Elliott et al. (2000) observed activation of the ventral striatum during reward presentation. Further, they were able to differentiate between those regions responding to a financial reward in general and those responding to high rewards occurring in the context of an increasing reward. Whereas the ventral striatum and subthalamic midbrain areas were sensitive to a general reward, the globus pallidus, the thalamus, and cingulate regions responded to the increasing rewards. Knutson et al. (2000) applied a reaction time task where the subjects could win or lose money depending on their task performance. In a control condition the performance had no consequence. During both reward and punishment, a significant activation of striatal (caudate and putamen) and thalamic regions as well as parts of the insula was found.

While these investigations focused on the effects of reward delivery, there also have been experiments on reward anticipation. This first phase of the reward processing is strongly linked with the concept of incentive motivation (Bolles, 1972). As could be demonstrated in animal studies, the most important brain structures involved in the anticipation of reward are dopaminergic and include midbrain regions (substantia nigra, ventral tegmental area) projecting to the striatum (nucleus caudatus, putamen, ventral striatum, in particular the nucleus accumbens) and the frontal cortex Schultz, 1998, Hollerman and Schultz, 1998, Ikemoto and Panksepp, 1999.

Recently, fMRI studies with human subjects could substantiate the importance of the aforementioned brain areas and the nucleus accumbens in particular for the anticipation of a reward. O'Doherty et al. (2002) used a cue paradigm with a visual stimulus as the cue and a taste stimulus as the reward. When analysing the anticipatory phase between cue and reward delivery they found activation in the ventral tegmental area, the substantia nigra, and the striatum including the nucleus accumbens. Knutson et al. (2001) investigated the anticipation of a monetary reward with a so-called monetary incentive delay task. In an event-related fMRI design, the authors revealed an increase of nucleus accumbens activation during reward but not during punishment anticipation. Furthermore, this activation was correlated with both the amount of monetary reward and the reported subjective happiness of the subjects. In contrast, medial caudate regions were involved during reward as well as punishment anticipation.

In both of the described fMRI studies on reward anticipation, the interval between the cue and the reward delivery was variable. Therefore, the subjects were not able to exactly foresee the reward onset. This type of unpredictability is suggested to be a necessary condition for the activation of the brain reward system Schultz et al., 1997, Hollerman and Schultz, 1998, Mirenowicz and Schultz, 1994.

Within the present investigation we attempted to fulfil the unpredictability criterion in a different manner. The temporal uncertainty with regard to the reward delivery was removed from our design, which consisted of a differential conditioning paradigm with a fixed cue-reward interval. Instead, we varied the reinforcer value by introducing response-dependent reward intensities. The subjects had to execute a reaction time task serving as the unconditioned stimulus (US). Depending upon their performance, the subjects could either win (or not win) money in the first condition. In a second condition, they were presented with a verbal feedback consisting of being fast or slow, whereas in a third control condition no task had to be executed at all. Each of the three conditions was introduced by a different visual cue (CS).

In accordance with the relevant literature mentioned before Knutson et al., 2001, O'Doherty et al., 2002, we predicted an activation of the reward system (the striatum, the ventral tegmental area, the substantia nigra, and in particular the nucleus accumbens) and additional reward-sensitive brain areas (the thalamus, the insula, and the orbitofrontal cortex) during the presentation of a CS that announces a rewarded reaction time task, compared to a CS announcing no task. Further, the anticipation of being able to obtain a monetary reward should be more motivating than the anticipation of a simple verbal feedback and therefore produce stronger activation in these regions.

Section snippets

Subjects

Twenty-seven right-handed subjects (24 females, mean age M = 23.3 years) were recruited from the current psychology undergraduates at the University of Giessen. The large number of female subjects reflects the high percentage of women in our undergraduate population. All subjects received course credit and were paid 10 €. In addition, the subjects could increase this amount by gaining additional money during the course of the experiment. All subjects gave their informed consent prior to

Results

All subjects participating in the experiment won money, on average € 10.89 (range € 3.00–13.30, SD 2.15). The number of task omissions was rather small, indicating a sufficient task involvement. The mean omission rates of the two reaction time conditions were comparable (mCS+: M = 0.29, vCS+: M = 0.37) which had been tested with a paired t test; 17 subjects had none, 6 subjects had one, 2 subjects two, and 2 more subjects four omissions.

Since we had predicted that the motivational variation of

Discussion

The present study investigated whether structures of the brain reward system can be activated during the anticipation of a reward and whether this activation varies with the motivational value of the reinforcer.

The results of the computed fixed effects model and the conjunction analysis showed that regions, which previously have been identified as responsive to reward delivery, are also involved in reward anticipation. The target regions (nucleus accumbens, putamen, globus pallidus, substantia

Acknowledgements

We thank Jay Schienle for his help editing the manuscript and two anonymous reviewers for their very helpful suggestions on an earlier version of this paper. This work was supported in part by a young scientist grant from the University of Giessen to P.K.

References (36)

  • C. Buechel et al.

    Amygdala-hippocampal involvement in human aversive trace conditioning revealed through event-related functional magnetic resonance imaging

    J. Neurosci.

    (1999)
  • T.J. Bussey et al.

    Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the ratImplications for the neurobiology of emotion

    Behav. Neurosci.

    (1997)
  • C.S. Carter et al.

    Anterior cingulate cortex, error detection, and the online monitoring of performance

    Science

    (1998)
  • H.D. Critchley et al.

    Cerebral correlates of autonomic cardiovascular arousala functional neuroimaging investigation

    J. Physiol.

    (2000)
  • M.R. Delgado et al.

    Tracking the hemodynamic responses to reward and punishment in the striatum

    J. Neurophysiol.

    (2000)
  • R. Elliott et al.

    Dissociable neural responses in human reward systems

    J. Neurosci.

    (2000)
  • S. Francis et al.

    The representation of pleasant touch in the brain and its relationship with taste and olfactory areas

    Neuroreport

    (1999)
  • M. Fredrikson et al.

    Functional neuroanatomical correlates of electrodermal activitya positron emission tomographic study

    Psychophysiology

    (1998)
  • Cited by (0)

    View full text