An fMRI study investigating cognitive modulation of brain regions associated with emotional processing of visual stimuli

Abstract

Brain regions modulated by cognitive tasks during emotional processing were investigated using fMRI. Participants performed indirect and direct emotional processing tasks on positive and negative faces and pictures. We used a multivariate technique, partial least squares (PLS) to determine spatially distributed patterns of brain activity associated with different tasks and stimulus conditions, as well as the interaction between the two. The pattern of brain activity accounting for the most task-related covariance represented a task×stimulus interaction and distinguished indirect processing of pictures and direct processing of faces from direct processing of pictures and indirect processing of faces. The latter two conditions were characterised by limbic (e.g. amygdala, insula, thalamus) and temporal lobe activity, in addition to greater activity in the ventral prefrontal cortex. Indirect and direct processing of pictures and faces, respectively, were represented by more dorsal prefrontal and parietal activity. These findings indicate that brain activity during processing of emotional content is critically dependent on both the type of stimulus and processing task. In addition, these results support the idea that the pattern of activity in the emotional network can be influenced in a ‘top–down’ fashion via cognitive factors such as attentional control, and as such, have important clinical implications for emotional disorders, such as depression and anxiety.

Introduction

Neuroimaging techniques such as positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have been used to map brain regions associated with the visual processing of emotional stimuli in healthy adults. Most of these studies have used either emotional faces or pictures in a variety of paradigms that manipulate subjects’ attention and physiological response to the emotional content of the stimuli in order to isolate activity. A major theme concerns the role of the amygdala, and although this brain region is important for certain types of emotional processing, some studies failed to find evidence for its involvement. This variability may be because the research strategies have been narrowly linked to particular tasks and stimuli. The type of task and the stimuli utilised are very important, as different aspects of emotional processing have been shown to modulate different brain regions. There has been no attempt to date, however, to combine different types of stimuli as well as emotional processing tasks to determine how stimulus features and processing task together affect activity in brain regions important for emotion, including the amygdala. This issue is the subject of the current experiment.

Damage to the amygdala causes reduced behavioural responses to emotional stimuli [3], [4], [5], [27]. For example, Adolphs and colleagues [4] presented pictures of faces conveying different emotional expressions to a patient (S.M.) who had selective bilateral damage to the amygdala. S.M. rated expressions of fear, anger and surprise as less intense than did brain-damaged controls, and exhibited severe recognition impairments specific to fear. Deficits in recognising negative facial expressions also are characteristic of patients with right hemisphere lesions in the primary or higher-order sensory cortices, particularly in the right inferior parietal cortex and right occipital cortex [1], [2].

Neuroimaging experiments also have shown the amygdala to be involved in processing facial expressions of emotion, particularly negative emotions, such as sadness [7] and fear [8], [32], [33]. Morris and co-workers [32], [33] found that the intensity of fearful facial expressions resulted in a monotonic increase in left amygdala activation. However, Kesler-West et al. [22] found amygdala activation during neutral face processing, with no additional increase in activation during the processing of emotional expressions, suggesting that the amygdala may be responsive to faces per se.

In support of the idea that the amygdala is sensitive to faces per se, Phillips et al. [37] found increased right amygdala response to neutral faces with time, in addition to more rapid decreases of right than left amygdala response to fearful faces. Some research suggests that there may be a laterality effect for amygdala activity during emotional face processing, with the left amygdala being active most consistently (e.g. [32], [33]), and it has been suggested that neutral faces may become conditioned stimuli to the right amygdala when associated with the presentation of fearful faces [37]. Right amygdala activity has been observed during conditioned fear acquisition in both animals and humans [14], [23], [27], [34]. Thus, it may be that the left amygdala is more involved in emotional face perception, and that the right amygdala is more involved in conditioned fear responses.

Subjects’ awareness of the emotional content of stimuli also influences amygdala activity during emotional face processing. Critchley et al. [11] found that explicit processing of emotional faces (judging facial expression) in an fMRI study was associated with increased activity in the temporal lobes, while implicit processing (determining gender) evoked greater activity in the amygdala. Similarly, activity in the amygdala to fearful faces has been found both when subjects attend to the faces in order to match identity or attend to other stimuli and ignore the faces [42]. In another study, Whalen et al. [43] found increased signal in the amygdala in response to masked fearful faces which subjects did not even report seeing. These findings, and that of Kesler-West et al. [22] as well as Winston et al. [44], suggest that increased activity in the amygdala during processing of faces may depend on various factors, including emotional valence and type of task, and may not require that attention be focused on the emotional expression itself or even on the face.

Activity in the brain regions involved in processing facial expression also is modulated by task demands. In one study [16] subjects were required to make an incidental (gender) or explicit (valence) decision about faces portraying neutral, happy or disgusted expressions. Activation of left inferior frontal and bilateral occipital–temporal regions was common to all conditions, whereas explicit judgements of disgust were associated with activity in the left amygdala and explicit judgements of happiness were characterised by bilateral orbitofrontal cortex activity. Hariri et al. [20] found that matching angry expressions increased activity in the amygdala bilaterally, while labelling expressions was associated with decreased activity in the same regions. They interpreted this finding as evidence that brain activity in limbic regions is modulated by higher brain regions (e.g. prefrontal cortex) via intellectual processes such as labelling. It may be that cognitive processing of a facial expression, such as would be necessary for attaching a verbal label to it, reduces the level of arousal associated with perception of a potentially threatening stimulus such as an angry face.

Amygdala activity during emotional picture processing has been less consistent than that seen for faces, and appears to be influenced by arousal factors in addition to valence. For example, Lane et al. [24] found amygdala activation during processing of both negative and positive pictures taken from the International Affective Picture System (IAPS: a standardised set of images designed to elicit a range of emotional responses, [26]), but only when highly arousing pictures were contrasted with low arousal pictures. A subsequent study in which participants, during scanning, focused on their subjective emotional response to the emotional pictures failed to generate an amygdala response but did elicit increased activity in anterior cingulate and medial prefrontal regions [24]. A relation between activity in the amygdala during viewing emotional pictures, the level of emotional arousal experienced by the individual and the ability to remember the pictures also has been demonstrated [10].

Structures outside of the limbic system also are activated during emotional processing of pictures and these responses are independent of the complexity of the images. Taylor et al. [41] found that viewing aversive images from the IAPS modulated occipital and occipital–temporal cortex activity relative to viewing non-aversive images. Increased activity in the thalamus, hypothalamus, midbrain and medial prefrontal cortex also is associated with emotional processing of pictures from the IAPS [25]. More recently, Liberzon et al. [28] investigated the effect of rating the pictures for aversive content or performing a recognition test for previously presented pictures on limbic activity. Limbic activation was greater during the rating versus the recognition condition, supporting the proposal that task demands can modify limbic activity during emotional processing. Furthermore, these results show that this modulation can occur with emotional pictures as well as emotional faces, but rating or labelling pictures results in increased limbic activity, whereas rating or labelling faces decreases limbic activity.

Considered as a whole, the evidence suggests that the processing of emotional visual stimuli recruits a number of limbic areas, such as the amygdala and insula, and non-limbic areas, including ventral and medial prefrontal cortices. The active regions appear to be influenced by the type of stimulus as well as the manner in which the stimuli are processed. However, the importance of these two factors has not been examined directly in the same individuals. Most studies have isolated the stimuli in order to localise brain activity associated with a specific valence and type of stimulus. As a result, the neural networks associated with processing positive and negative faces and pictures, in isolation, are at least partially understood, as are the regions modulated by cognitive task demands for each type of stimulus. What is unknown is whether these networks are similar across stimulus types, and whether the interaction between stimulus type and processing task will have an effect on related brain activity.

Accordingly, the current experiment was designed to (1) compare brain activity directly during the processing of emotional faces and pictures, (2) to assess the influence of processing task on brain activity, and (3) to investigate the neural correlates associated with the interaction between stimulus type and processing task. Participants viewed pictures of faces or scenes that portrayed either positive or negative emotional expressions or content. Task instructions focused attention on the emotional content of the stimuli (the direct emotional conditions) or on other aspects of the stimuli (the indirect conditions). One potential interaction of stimulus and processing type that we expected based on earlier results [10], [11], was that activity in the amygdala would be greater for faces during indirect processing but greater for pictures during direct processing, where arousal appears to be greater than during other types of cognitive tasks [28].

In addition to examining the amygdala, we studied the spatial distribution of activity across the brain in order to identify how whole brain patterns are modified by type of emotional stimulus and processing demands. We also examined the functional connectivity of the amygdala during direct and indirect emotional processing to clarify how the amygdala interacts with other brain regions during emotional processing. Functional connectivity refers to the way activity in a given region is correlated with activity in other areas of the brain during an experimental condition of interest [31]. Studies of functional connectivity have shown that the inter-regional correlations of a brain region can differ across conditions or groups [17] even if the mean activity in that region does not vary, and are useful for exploring the distributed neural networks underlying cognitive processes.