Regular articleAnticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system:: an event-related fMRI study
Introduction
Brain regions that are responsive to reward delivery have been studied intensely over recent years. Findings from animal studies point to the important role of the striatum, the orbitofrontal cortex Rolls, 1999, Schultz et al., 2000, the thalamus Oyoshi et al., 1996, Komura et al., 2001, and the anterior cingulate Bussey et al., 1997, Parkinson et al., 2000 for the processing of reward stimuli.
In humans comparable structures have been identified by using modern neuroimaging techniques such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET). Here, the application of primary reinforcers (e.g., pleasant taste, touch, or smell) led to an activation of the orbitofrontal and prefrontal cortex, the insula, the anterior cingulate as well as the striatum Francis et al., 1999, O'Doherty et al., 2002. The presentation of secondary reinforcers provoked similar activation patterns. In a study by Thut et al. (1997), the prefrontal, midbrain, and thalamic activation increased when correct responses in a go/nogo task were rewarded with money compared to verbal reinforcement. Delgado et al. (2000) used a gambling task, where subjects could win or lose money. Both conditions provoked activation in the bilateral caudate and the left ventral striatum. In another type of gambling task, Elliott et al. (2000) observed activation of the ventral striatum during reward presentation. Further, they were able to differentiate between those regions responding to a financial reward in general and those responding to high rewards occurring in the context of an increasing reward. Whereas the ventral striatum and subthalamic midbrain areas were sensitive to a general reward, the globus pallidus, the thalamus, and cingulate regions responded to the increasing rewards. Knutson et al. (2000) applied a reaction time task where the subjects could win or lose money depending on their task performance. In a control condition the performance had no consequence. During both reward and punishment, a significant activation of striatal (caudate and putamen) and thalamic regions as well as parts of the insula was found.
While these investigations focused on the effects of reward delivery, there also have been experiments on reward anticipation. This first phase of the reward processing is strongly linked with the concept of incentive motivation (Bolles, 1972). As could be demonstrated in animal studies, the most important brain structures involved in the anticipation of reward are dopaminergic and include midbrain regions (substantia nigra, ventral tegmental area) projecting to the striatum (nucleus caudatus, putamen, ventral striatum, in particular the nucleus accumbens) and the frontal cortex Schultz, 1998, Hollerman and Schultz, 1998, Ikemoto and Panksepp, 1999.
Recently, fMRI studies with human subjects could substantiate the importance of the aforementioned brain areas and the nucleus accumbens in particular for the anticipation of a reward. O'Doherty et al. (2002) used a cue paradigm with a visual stimulus as the cue and a taste stimulus as the reward. When analysing the anticipatory phase between cue and reward delivery they found activation in the ventral tegmental area, the substantia nigra, and the striatum including the nucleus accumbens. Knutson et al. (2001) investigated the anticipation of a monetary reward with a so-called monetary incentive delay task. In an event-related fMRI design, the authors revealed an increase of nucleus accumbens activation during reward but not during punishment anticipation. Furthermore, this activation was correlated with both the amount of monetary reward and the reported subjective happiness of the subjects. In contrast, medial caudate regions were involved during reward as well as punishment anticipation.
In both of the described fMRI studies on reward anticipation, the interval between the cue and the reward delivery was variable. Therefore, the subjects were not able to exactly foresee the reward onset. This type of unpredictability is suggested to be a necessary condition for the activation of the brain reward system Schultz et al., 1997, Hollerman and Schultz, 1998, Mirenowicz and Schultz, 1994.
Within the present investigation we attempted to fulfil the unpredictability criterion in a different manner. The temporal uncertainty with regard to the reward delivery was removed from our design, which consisted of a differential conditioning paradigm with a fixed cue-reward interval. Instead, we varied the reinforcer value by introducing response-dependent reward intensities. The subjects had to execute a reaction time task serving as the unconditioned stimulus (US). Depending upon their performance, the subjects could either win (or not win) money in the first condition. In a second condition, they were presented with a verbal feedback consisting of being fast or slow, whereas in a third control condition no task had to be executed at all. Each of the three conditions was introduced by a different visual cue (CS).
In accordance with the relevant literature mentioned before Knutson et al., 2001, O'Doherty et al., 2002, we predicted an activation of the reward system (the striatum, the ventral tegmental area, the substantia nigra, and in particular the nucleus accumbens) and additional reward-sensitive brain areas (the thalamus, the insula, and the orbitofrontal cortex) during the presentation of a CS that announces a rewarded reaction time task, compared to a CS announcing no task. Further, the anticipation of being able to obtain a monetary reward should be more motivating than the anticipation of a simple verbal feedback and therefore produce stronger activation in these regions.
Section snippets
Subjects
Twenty-seven right-handed subjects (24 females, mean age M = 23.3 years) were recruited from the current psychology undergraduates at the University of Giessen. The large number of female subjects reflects the high percentage of women in our undergraduate population. All subjects received course credit and were paid 10 €. In addition, the subjects could increase this amount by gaining additional money during the course of the experiment. All subjects gave their informed consent prior to
Results
All subjects participating in the experiment won money, on average € 10.89 (range € 3.00–13.30, SD 2.15). The number of task omissions was rather small, indicating a sufficient task involvement. The mean omission rates of the two reaction time conditions were comparable (mCS+: M = 0.29, vCS+: M = 0.37) which had been tested with a paired t test; 17 subjects had none, 6 subjects had one, 2 subjects two, and 2 more subjects four omissions.
Since we had predicted that the motivational variation of
Discussion
The present study investigated whether structures of the brain reward system can be activated during the anticipation of a reward and whether this activation varies with the motivational value of the reinforcer.
The results of the computed fixed effects model and the conjunction analysis showed that regions, which previously have been identified as responsive to reward delivery, are also involved in reward anticipation. The target regions (nucleus accumbens, putamen, globus pallidus, substantia
Acknowledgements
We thank Jay Schienle for his help editing the manuscript and two anonymous reviewers for their very helpful suggestions on an earlier version of this paper. This work was supported in part by a young scientist grant from the University of Giessen to P.K.
References (36)
Circuitry and functional aspects of the insular lobe in primates including humans
Brain Res. Brain Res. Rev.
(1996)- et al.
Neural activity in the human brain relating to uncertainty and arousal during anticipation
Neuron
(2001) - et al.
Analysis of fMRI time-series revisited
Neuroimage
(1995) - et al.
The role of nucleus accumbens dopamine in motivated behaviora unifying interpretation with special reference to reward-seeking
Brain Res. Rev.
(1999) - et al.
FMRI visualization of brain activity during a monetary incentive delay task
Neuroimage
(2000) - et al.
Neural responses during anticipation of a primary taste reward
Neuron
(2002) - et al.
Deficits in visual cognition and attention following bilateral anterior cingulotomy
Neuropsychologia
(2001) - et al.
Frontal lesions and electrodermal activityeffects of significance
Neuropsychologia
(1999) - et al.
Nonlinear spatial normalization using basis functions
Hum. Brain Mapp.
(1999) Reinforcement, expectancy, and learning
Psychol. Rev.
(1972)