Elsevier

NeuroImage

Volume 24, Issue 2, 15 January 2005, Pages 417-425
NeuroImage

Differential amygdala responses to happy and fearful facial expressions depend on selective attention

https://doi.org/10.1016/j.neuroimage.2004.08.017Get rights and content

Facial expressions of emotion elicit increased activity in the human amygdala. Such increases are particularly evident for expressions that convey potential threat to the observer, and arise even when the face is masked from awareness. We used functional magnetic resonance imaging (fMRI) to examine whether the amygdala responds differentially to threatening (fearful) versus nonthreatening (happy) facial expressions depending on whether the face is attended or actively ignored. In separate runs, participants were cued to attend to a face or a house within semitransparent, spatially overlaid composite pairs, presented either side of fixation, and were required to perform a demanding same/different judgment. We found significant attentional modulation of activity in category-specific ‘face’ (fusiform gyrus) and ‘place’ (parahippocampal gyrus) regions, with activity in each area increasing selectively when its preferred stimulus was attended versus ignored. In contrast, activity in the amygdala differed according to the valence of the facial expression and the category of the attended stimulus. For happy faces, activity in the amygdala was greater in the attend-face than in the attend-house condition, whereas for fearful faces, activity was greater in the attend-house than in the attend-face condition. We conclude that differential amygdala responses to fearful versus happy facial expressions are tuned by mechanisms of attention and that the amygdala gives preference to potentially threatening stimuli under conditions of inattention.

Introduction

An ability to perceive threatening stimuli automatically may provide an advantage in warning of potential danger (Adolphs, 2002). Perception of positive or rewarding stimuli, on the other hand, is only relevant in a social context and so may rely less on rapid processing mechanisms. The human face is an important source of information concerning both threat-related and nonthreatening (social) environmental stimuli. Studies that have examined brain activity associated with perception of facial expressions have shown that the amygdala responds selectively to a range of emotions, particularly those that signal potential threat such as fear and anger (Adolphs, 2002). Although several studies have shown that the amygdala responds selectively to emotive facial expressions, even when these are not consciously perceived (Morris et al., 1998, Morris et al., 2001, Whalen et al., 1998, Williams et al., 2004; though see Phillips et al., 2004), there is conflicting evidence on whether the amygdala continues to respond to facial expressions under conditions of inattention. Some studies using functional magnetic resonance imaging (fMRI) have shown that amygdala activity in response to facial expressions is maintained when focal attention is directed toward a distractor stimulus (and thus away from the face) (Anderson et al., 2003, Vuilleumier et al., 2001); others have reported significant decreases in amygdala activity under conditions of reduced attention (Holmes et al., 2003, Pessoa et al., 2002), consistent with findings of attentional modulation in extrastriate visual areas (Kanwisher and Wojciulik, 2000). Thus, despite evidence that the amygdala responds to faces that are not consciously perceived, the circumstances under which attention influences amygdala activity remain unclear.

There are several possible reasons for the discrepant findings on the effects of attention on amygdala responses to affective faces. One possibility concerns the attentional demands of the task used to divert processing resources from face stimuli. In a widely cited study, Vuilleumier et al. (2001) presented pairs of face and house stimuli either to the left and right, or above and below, central fixation. They had participants focus their attention on either the horizontal or vertical pairs of stimuli to make a same/different judgment. Amygdala activity remained higher for fearful than neutral faces even when the house stimuli were attended and the face stimuli ignored, implying that amygdala responses to fear are maintained under conditions of inattention. A potential limitation of this study, however, is that the distractor task—focusing on the houses rather than on the faces—was relatively easy and may thus have left residual attentional capacity for processing of the faces.

A stronger test of the hypothesis that amygdala responses to facial expressions are maintained under conditions of inattention requires a distractor task that is sufficiently demanding of attentional resources that there is little or no residual capacity available for processing the face stimuli. This notion arises from the seminal work of Lavie, 1995, Lavie, 2000 who has suggested that under conditions of low perceptual load, resources that are not used in processing a target stimulus ‘spill over’ to distractors; and that when attentional resources are completely exhausted by the target task, any distractors will not be processed. In a recent fMRI study, Pessoa et al. (2002) addressed this hypothesis by having participants perform a difficult orientation-discrimination task on a pair of peripheral line segments that appeared simultaneously with a face at fixation. Activity in the amygdala decreased significantly in response to fearful and happy faces during the peripheral attention task, relative to conditions in which participants ignored the line segments and focused their attention instead on the central faces.

The study of Pessoa et al. (2002) raises doubts about the conclusion that the human amygdala responds to threat-related facial affect under conditions of inattention (cf. Vuilleumier et al., 2001). However, data from a recent series of behavioral studies have raised the question of whether face stimuli are themselves susceptible to attentional load in the same way as ‘nonbiological’ stimuli such as letters and words (Lavie et al., 2003). It therefore remains an open question whether manipulations of attentional load should affect amygdala responses to emotional expressions conveyed by the human face.

A further possible reason for the discrepant findings on the effects of attention on amygdala responses to facial affect concerns the different display configurations used in previous investigations. In the study of Vuilleumier et al. (2001), face stimuli appeared peripherally, whereas Pessoa et al. (2002) presented faces at fixation. It is known that the human amygdala responds preferentially to low spatial frequency signals conveyed via the magnocellular visual pathway (Vuilleumier et al., 2003), and this pathway in turn is driven maximally by inputs from the visual periphery (Kandel et al., 2000). It is therefore likely that faces presented peripherally, as in the study by Vuilleumier et al. (2001), will result in stronger amygdala responses than faces presented at the fovea (cf. Pessoa et al., 2002). Robust amygdala responses to peripheral faces may thus be less susceptible to attentional modulation, a possibility that is consistent with the hypothesized role of the amygdala in threat detection. This idea is further complicated, however, by the recent findings of Anderson et al. (2003), who used composite stimuli consisting of overlaid images of a face and a house at fixation. When participants focused on the house, amygdala activation to a fearful face remained unchanged relative to conditions in which the fearful face was attended. These findings suggest that under some circumstances even foveal inputs are resilient to modulation by attention.

Finally, previous imaging studies that have shown sustained amygdala responses to facial expressions under conditions of inattention (Anderson et al., 2003, Vuilleumier et al., 2001) used randomized, event-related designs, so that participants were not aware of the facial expression or task until just before stimulus presentation. In contrast, the study of Pessoa et al. (2002) utilized a blocked design so that when participants were instructed to attend to the peripheral line segments, there was sufficient time for activation of cognitive processes underlying this task. The observed effect of these attentional processes was a decrease of baseline activity in the amygdala, presumably reflecting active suppression of the competing fearful faces so that they would not interfere with performance on the peripheral discrimination task.

The aim of the present study was to examine attentional modulation of amygdala activity to facial expressions using a task that uniquely combined elements of several of the studies reviewed above (Anderson et al., 2003, Pessoa et al., 2002, Vuilleumier et al., 2001). First, we used spatially overlaid face and house stimuli (see Fig. 1a) so that features rather than locations would be more important for segregating attended and ignored images (O'Craven et al., 1999). Selectively attending to a single stimulus class in such ‘rivalrous’ conditions has been shown to selectively activate specialized extrastriate cortical areas (O'Craven et al., 1999). Moreover, such displays afford an opportunity to isolate regions of interest (ROI) for investigating any modulation of face- and house-specific cortical activity by attention. Second, we used displays in which pairs of face/house composites appeared in the visual periphery, thus biasing activity in favor of magnocellular inputs and minimizing the possibility that any effect of attentional modulation might be attributed to a reduced magnocellular input for foveal stimuli. Third, we had participants perform a highly demanding discrimination task on each pair of composite face/house stimuli and verified their performance behaviorally to ensure maximal attentional load. Fourth, we used a blocked design rather than an event-related design to ensure sufficient time and predictability for attention to be effectively deployed. Finally, it has been suggested that information conveyed via the magnocellular visual pathway is relatively crude and that the amygdala cannot therefore distinguish between different facial expressions under conditions of reduced attention (Anderson et al., 2003). According to this view, any facial expression that is actively ignored should result in increased activity in the amygdala, not just expressions that signal potential threat. To test this hypothesis, we compared amygdala activity in response to a happy facial expression (a positive or rewarding stimulus) under attended and ignored conditions. Fearful faces were presented in a separate experimental run to examine the effect of attention on amygdala responses to threat-related stimuli, as in previous investigations (Anderson et al., 2003, Pessoa et al., 2002, Vuilleumier et al., 2001).

In two initial behavioral experiments, we verified that attending to the house images in the composite face/house displays had a detrimental effect on perception of the faces. In Experiment 1, separate blocks of trials contained faces with happy or neutral expressions. In Experiment 2, separate blocks contained faces with fearful or neutral expressions. In each experiment, participants were first asked to attend to the faces within each horizontally aligned pair of composite stimuli and to make a same/different identity judgment (single task). In a second set of trials, they were asked to judge whether the houses in each composite pair were the same or different, and then to do the same for the faces to the best of their ability (dual task). Note that the single task indexed performance for face perception with full attention, whereas the dual task indexed the extent of any detrimental effect of the house-judgment task on face perception. This comparison was made separately for both happy and fearful expressions. If, as Anderson et al. (2003) suggest, inputs to the amygdala via the magnocellular pathway do not differentiate between facial expressions, then happy and fearful faces should be equally affected under the dual task conditions. If, on the other hand, the amygdala responds only to fearful stimuli, then ignoring the faces in the dual task should have little effect on performance for fearful but not for happy expressions.

Following our behavioral experiments, we used fMRI to investigate the neural correlates of affective face perception under conditions of inattention. A series of localizer scans enabled independent determination of regions of interest (ROIs) for subsequent analyses of the experimental runs. In separate blocks, participants viewed (nonoverlapping) pairs of faces, houses, scrambled objects, or placeholders, and performed a simple same/different judgment on each. We focused on several extrastriate cortical areas that have previously been described: the fusiform face area (FFA), which responds preferentially to faces relative to other stimuli (Kanwisher et al., 1997); the superior temporal sulcus (STS), which has been suggested to encode biologically relevant aspects of faces such as affect (Narumoto et al., 2001); and the parahippocampal place area (PPA), which responds preferentially to places or scenes relative to other stimuli (Epstein et al., 1999). Focusing our analyses on these independently defined regions allowed assessment of the effectiveness of our attentional manipulation. Previous studies have demonstrated increased activation of the FFA when faces are attended, and significant decreases when faces are ignored (O'Craven et al., 1999, Wojciulik et al., 1998). Similarly, studies have shown increased activation of the PPA when house stimuli are the focus of attention, and decreases when they are ignored (O'Craven et al., 1999, Wojciulik et al., 1998). In the absence of a widely accepted standard for functional localization of the amygdala, we defined the amygdaloid complex in each hemisphere using an anatomically defined region of interest.

Following the localizer runs, participants were scanned while viewing the composite face/house stimulus pairs used in the behavioral tasks (Experiments 1 and 2, described above). In contrast to the behavioral experiments, however, participants were only ever required to do a single task during scanning: either the attend house task, in which they indicated whether the houses were the same or different; or the attend face task, in which they indicated whether the identities of the faces were the same or different. Blocks of each task were interleaved throughout a scanning run. There were two scanning runs, one in which the faces displayed happy or neutral expressions (in separate blocks), and one in which the faces showed fearful or neutral expressions (also in separate blocks). This design allowed examination of the effects of attention on activity in the amygdala for the two different expressions independently.

Consistent with predictions for behavioral experiments, we hypothesized that if inputs from the magnocellular pathway are indeed too crude to allow the amygdala to distinguish between different expressions (Anderson et al., 2003), then there should be no difference in the effect of attention on the pattern of amygdala responses to happy and fearful expressions during the attend house task. Conversely, if information conveyed via the magnocellular pathways is able to support discrimination between expressions, then there may be sustained activity in the amygdala for fearful but not happy faces under conditions of reduced attention due to the importance of fearful expressions in signaling potential threat. A third alternative is that regardless of facial expression, attention could cause an overall reduction in amygdala activity, consistent with the effect documented by Pessoa et al. (2002).

Section snippets

Participants

Thirteen individuals (nine male, four female; mean age = 28 years, SD = 4.08) gave written, informed consent to participate according to the guidelines of the Human Research Ethics Committee at the University of Melbourne. All participants were right-handed, as assessed by the Edinburgh Handedness Inventory; all were classified as neurologically normal by medical review; and all had normal or corrected-to-normal visual acuity.

Materials

Visual stimuli were presented as black and white displays on a

Behavioral data

The behavioral tasks were undertaken to investigate whether selective attention to one stimulus category within a composite pair (face or house) impaired perception for the other category. As shown in Fig. 1b, the percentage correct data revealed that participants were better at determining whether the identities of the faces in each composite pair were the same or different in the single task (i.e., attend-face only) than in the dual task (t(12) = 7.29, P < 0.001; Fig. 1b). Interestingly,

Discussion

The principal aim of this study was to examine activity within the amygdala and ventral extrastriate areas in response to fearful and happy facial expressions during an attentionally demanding spatial-rivalry task. The FFA and PPA showed increased activity when their ‘preferred’ image type (face or house) was attended, relative to when the preferred image was ignored. These findings are consistent with reports of attention modulating face-specific activity in the fusiform gyrus and

Acknowledgments

This work was supported by a grant from Unilever Research, UK. We would like to acknowledge a contribution toward scan costs made by the Brain Imaging Research Foundation. We would also like to thank Chris Chambers, Ross Cunnington, Matthew Howard, Adam Morris, and Anina Rich for their comments and suggestions on this work.

References (28)

  • P. Ekman et al.

    Pictures of Facial Affect

    (1975)
  • E.R. Kandel et al.

    Principles of Neural Science

    (2000)
  • N. Kanwisher et al.

    Visual attention: insights from brain imaging

    Nat. Rev., Neurosci.

    (2000)
  • N. Kanwisher et al.

    The fusiform face area: a module in human extrastriate cortex specialized for face perception

    J. Neurosci.

    (1997)
  • Cited by (0)

    View full text