Elsevier

NeuroImage

Volume 42, Issue 4, 1 October 2008, Pages 1714-1727
NeuroImage

Visual mental imagery and perception produce opposite adaptation effects on early brain potentials

https://doi.org/10.1016/j.neuroimage.2008.07.004Get rights and content

Abstract

Event-related potentials (ERPs) were recorded during a rapid adaptation paradigm to determine whether visual perception and visual mental imagery of faces recruit the same early perceptual processes. The early effect of face and object adaptors, either perceived or visualized, on test stimuli, was assessed by measuring the amplitude of the N170/VPP complex, typically much larger for faces than for other object categories. Faces elicited a robust N170/VPP complex, localized to posterior ventrolateral occipitotemporal cortex. Both visualized and perceived adaptors affected the N170/VPP complex to test faces from 120 ms post-stimulus, reflecting effects on neural populations supporting early perceptual face categorization. Critically, while perceived adaptors suppressed the amplitude of the N170/VPP, visualized adaptors enhanced it. We suggest that perceived adaptors affect neural populations in the neocortex supporting early perceptual processing of faces via bottom-up mechanisms, whereas visualized adaptors affect them via top-down mechanisms. Similar enhancement effects were found on the N170/VPP complex to non-face objects, suggesting such effects are a general consequence of visual imagery on processing of faces and other object categories.

These findings support image-percept equivalence theories and may explain, in part, why visual percepts and visual mental images are not routinely confused, even though both engage similar neural populations in the visual system.

Introduction

During visual mental imagery, neural representations of a visual entity are reactivated endogenously from long-term memory and maintained in working memory to be inspected and transformed, processes at the core of many common mental activities, such as spatial reasoning (e.g., Ganis et al., 2004, Kosslyn et al., 2006). A prominent class of theories (image-percept equivalence theories) postulates that visual mental images are supported by the same neural processes and representations underlying visual perception (e.g., Finke, 1985, Kosslyn, 1994, Kosslyn et al., 2006). The question of whether visual mental imagery and visual perception engage the same neural processes is important because it presents a dilemma: If the processes are the same in the two cases, then how can the brain distinguish visual percepts from visual mental images? Note that, although there is evidence that visual mental imagery of a stimulus can lead to false memories of having perceived the stimulus (e.g., Gonsalves et al., 2004), this situation is the exception, rather than the rule, and may relate more to issues of how prior experience is encoded and subsequently retrieved from long-term memory, than how the brain distinguishes between ongoing internal activation of mental representations and ongoing perception of the world.

Although image-percept equivalence theories do not postulate that visual mental images and percepts are identical in every detail, they predict that many psychophysical effects (and the underlying neural processes) found with actual visual stimuli should also be present when these stimuli are only visualized (Kosslyn et al., 2006). Consistent with this prediction, a number of behavioral studies have shown that aftereffect illusions typically brought about by visual perception can also be produced by visual mental imagery. For instance, early work (Finke and Schmidt, 1978) showed that imaging bars against a colored background produces orientation-specific effects on subsequently presented gratings (McCollough effect) similar to those produced by actually perceiving the bars (see for extensive review, Kosslyn et al., 2006). However, other researchers have used behavioral data to argue that these types of results are artifacts due to experimenter expectancy or demand artifacts (Broerse and Crassini, 1980, Broerse and Crassini, 1984, Intons-Peterson, 1983, Intons-Peterson and White, 1981, Pylyshyn, 1981). Findings from behavioral data in sophisticated cognitive paradigms can be used to make strong inferences, but behavioral measures alone cannot determine conclusively the exact processes underlying an effect because the pattern of behavioral effects (e.g., response times, error rates) is the cumulative result of processing at multiple levels in the brain. Differences potentially can arise at any one or more of these levels and potentially influence the next, and so on down the line.

Recently, cognitive neuroscience has used neuroimaging to try to provide more direct evidence on the issue by showing that visual mental imagery elicits brain activation in regions engaged during visual perception of the same stimuli (e.g., Ganis et al., 2004, Ishai et al., 2000, Ishai et al., 2002, Kosslyn et al., 1996, Mechelli et al., 2004, O'Craven and Kanwisher, 2000). For example, visualizing faces activates regions in inferotemporal cortex that are also activated while perceiving faces, whereas visualizing houses activates portions of the parahippocampal cortex that are also activated while perceiving houses (e.g., Ishai et al., 2002, O'Craven and Kanwisher, 2000). Functional magnetic resonance imaging (fMRI) results of this kind, however, are ambiguous, for two reasons. First, the limited temporal resolution of fMRI, relative to the rapid timing of neural processing, cannot distinguish between alternative time course scenarios. There is evidence that relatively early visual areas in the ventral stream participate not only in lower-level visual processing, but also in higher-level cognitive processing with a later time course (e.g., Lamme and Roelfsema, 2000). This late engagement may be supported by higher-level brain regions such as the anterior temporal and prefrontal cortices, which may exert top-down reactivation onto low-level visual areas over an extended processing time. Second, even ignoring the complex relationship between neural activity and hemodynamic responses (e.g., Logothetis and Wandell, 2004), neural populations may exhibit the same average activation but be in different functional states, due to their participation in different processes (Gilbert and Sigman, 2007, Schendan and Kutas, 2007). Such different functional states would not be evident in activation maps, but could be revealed by assessing how these neural populations react to subsequent probe stimuli.

The above considerations led us to address this question using scalp event-related potentials (ERPs) to monitor non-invasively neural adaptation effects on face processing induced by visual mental imagery and visual perception. Note that, although we focused on faces, the following ideas can be used to address the same question about other object categories as well. Neural adaptation, the phenomenon by which neural responses to a stimulus are modified (usually suppressed) by immediate repetition with a very short time delay between the first stimulus and the repeated stimulus (less than 500 ms) has been used to probe the properties of neural representations (e.g., Grill-Spector et al., 1999). The basic adaptation logic involves assessing neural activation to a test stimulus that is either preceded by an adaptor stimulus or not. The difference between the response elicited by non-adapted and adapted stimuli, respectively, is referred to as an adaptation effect. To study neural selectivity, the adaptor and test stimuli usually are identical, defining the maximal adaptation effect, and this is compared to a condition where the adaptor and test stimuli differ along one dimension of interest. If both adaptor stimuli produce similar effects on the test stimulus, it is concluded that the underlying neural representation is invariant for the manipulated dimension. For example, if the response of a neural population in the inferotemporal cortex to a test face is suppressed equally by presenting as adaptors either the same face or a larger face, then one can infer that this neural population implements a representation that is invariant to face size. By measuring neural processes with sufficient temporal resolution, the time course of this representational invariance can be monitored as stimulus processing unfolds. In humans, early adaptation effects with visual faces and objects have been reported in several studies using scalp ERPs (e.g., Itier and Taylor, 2002, Jeffreys, 1996, Kovacs et al., 2006), intracranial ERPs (e.g., Seeck et al., 1997), and event-related fields (ERFs) (e.g., Harris and Nakayama, 2007a, Noguchi et al., 2004), employing a broad range of adaptation durations and interstimulus intervals (ISI). Most studies focused on reduced amplitude, as a result of the pre-exposure to adaptor stimuli, of two related brain potentials elicited by faces peaking between 140 and 200 ms post-stimulus, the N170 and the vertex positive potential, VPP (also known as P150, e.g., Schendan et al., 1998). This N170/VPP complex is largest for faces than other objects, and thought to index early perceptual processing of faces (e.g., perceptual categorization of faces) implemented in ventrolateral temporal regions (e.g., Bentin et al., 1996, Jeffreys, 1989, Jeffreys, 1996, Joyce and Rossion, 2005, Schendan et al., 1998). Consequently, effects on the N170/VPP complex to faces are generally thought to show that the manipulation of interest affects early bottom-up perceptual processing of faces, as opposed to later stages (e.g., Henson et al., 2003, Trenner et al., 2004). Note, the literature has been somewhat confused about the consistency of early neural adaptation effects, due to collapsing results across a variety of repetition paradigms. Repetition paradigms may involve different neural mechanisms depending on (a) the duration of the stimuli, especially the adaptor, (b) the ISI between the first and repeated stimuli, and (c) whether intervening stimuli occur between first and repeated stimuli, and so studies investigating these different phenomena must be separated. For example, results showing no early effects of face repetition (e.g., Puce et al., 1999, Schweinberger et al., 2002b) using relatively long ISIs (∼ 2 s) are actually entirely compatible with findings showing early effects of repetition (e.g., Harris and Nakayama, 2007b, Jeffreys, 1996, Kovacs et al., 2006) with short ISIs (200 ms or less) because different mechanisms likely mediate each case (see Huber, 2008 for a recent discussion about the relationship between adaptation and repetition priming).

We used this adaptation logic, while monitoring neural activity with ERPs, to investigate: i) whether visual mental imagery of faces engages similar neural populations involved in the perceptual categorization of faces, and whether the time course is consistent with early perceptual processes, and ii) whether the effect of engaging these neural populations is the same in both cases, as assessed by responses to subsequent test stimuli. If visualized face adaptors affect the N170/VPP complex evoked by a test face stimulus, then we can conclude that visual mental imagery of faces recruits neural processes supporting early face perception. Conversely, lack of effects on the N170/VPP complex may suggest that visualized adaptors do not affect neural populations and processes engaged by early face perception. Furthermore, if the pattern of effects on the N170/VPP complex (e.g., direction, latency, or magnitude) is the same for perceived and visualized adaptors, then, even at this early stage, the neural processes engaged by visual mental imagery and visual perception overlap. In contrast, different patterns of N170/VPP effects would indicate that visual mental imagery and visual perception may both affect neural populations involved in early face perception, but use distinct neural pathways. Such differences would reveal potential mechanisms that enable the brain not to confuse visual percepts and visual mental images. We emphasize that, although this study focuses on test faces because they elicit the largest N170 and VPP components, we do not mean that our findings are specific for faces: The same logic and inferences apply also to other object categories. Finally, this study was focused on the N170/VPP, but, for completeness, we also analyzed later ERPs implicated in initial cognitive processing of faces and objects that have a polarity-inverting scalp distribution similar to the N170/VPP: An occipitotemporal N250/Ncl from 200 to 300 ms implicated in subordinate categorization, category learning, short-term repetition priming, and stored face representations (e.g., Doniger et al., 2000, Doniger et al., 2001, Henson and Rugg, 2003, Schweinberger et al., 2002b, Scott et al., 2006, Sehatpour et al., 2006), and N400-like components to faces and objects peaking from 300 to 500 ms implicated in categorization tasks, perceptual and conceptual knowledge, semantic priming, and long-term repetition priming (e.g., Barrett et al., 1988, Cooper et al., 2007, McPherson and Holcomb, 1999, Paller et al., 2007, Schendan and Kutas, 2002, Schendan and Kutas, 2007, Schweinberger et al., 2002a).

Section snippets

Subjects

A total of 23 naive healthy volunteers, recruited from Tufts University and the greater Boston area, took part in the study for course credit or cash (9 females and 14 males, average age: 21 years; standard deviation of age: 1.03 years). All subjects had normal or corrected vision, and no history of neurological or psychiatric disease. The data from 4 subjects were not included in the analyses because they had too few usable trials, due to excessive eye movement artifacts. The demographics of

Results

We begin with the results for face and object adaptors to define the basic category effect and show non-adapted baseline activity. Next, we present the ERPs to the perceptual adaptation conditions, followed by those to the imagery adaptation conditions. For brevity, we report only nontrivial effects of site and hemisphere (i.e., those interacting with the adaptor category factor).

Discussion

We begin by summarizing the results. The most important and novel findings of this study are that i) both visualized and perceived adaptors affect the N170/VPP complex with a similar time course, and ii) remarkably, while perceived adaptors suppress the amplitude of the N170/VPP complex, visualized adaptors enhance it.

Non-adapted face stimuli (i.e., faces used as adaptors at the beginning of a trial) evoke a larger N170/VPP complex than non-adapted object stimuli, replicating previous ERP work

Summary and conclusions

This is the first ERP study to compare the effect of visualized and perceived adaptors on the early cortical processing of faces. An adaptation paradigm was used to probe the effect of perceived and visualized faces and objects on face-sensitive neural populations. The first finding is that both visualized and perceived adaptors have an effect on the N170/VPP complex elicited by test faces. This effect onsets at 120 ms post-stimulus, indicating that both visual perception and visual mental

Acknowledgments

Authors listed in alphabetical order, and contributed equally to all phases of this project. Research was supported by Tufts University Faculty start-up funds to H.E.S who was also supported by a Faculty Research Award Committee (FRAC) Research Semester Fellowship from Tufts University. Data acquisition was conducted in the Vision and Memory Neuroimaging Lab in the Department of Psychology at Tufts University. We would like to thank Stephen M. Maher, Lisa C. Lucia, and Roderick Elias for their

References (80)

  • IshaiA. et al.

    Distributed neural systems for the generation of visual images

    Neuron

    (2000)
  • IshaiA. et al.

    Visual imagery of famous faces: effects of memory and attention revealed by fMRI

    Neuroimage

    (2002)
  • ItierR.J. et al.

    Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs

    Neuroimage

    (2002)
  • JoyceC. et al.

    The face-sensitive N170 and VPP components manifest the same brain processes: the effect of reference electrode site

    Clin. Neurophysiol.

    (2005)
  • LammeV.A. et al.

    The distinct modes of vision offered by feedforward and recurrent processing

    Trends Neurosci.

    (2000)
  • PallerK.A. et al.

    Validating neural correlates of familiarity

    Trends Cogn. Sci.

    (2007)
  • SchendanH.E. et al.

    Neurophysiological evidence for two processing times for visual object identification

    Neuropsychologia

    (2002)
  • SchendanH.E. et al.

    Mental rotation and object categorization share a common network of prefrontal and dorsal and ventral regions of posterior cortex

    Neuroimage

    (2007)
  • SchonK. et al.

    Delayed match to object or place: an event-related fMRI study of short-term stimulus maintenance and the role of stimulus pre-exposure

    Neuroimage

    (2008)
  • SchweinbergerS.R. et al.

    Human brain potential correlates of repetition priming in face and name recognition

    Neuropsychologia

    (2002)
  • SchweinbergerS.R. et al.

    Event-related brain potential evidence for a response of inferior temporal cortex to familiar face repetitions

    Brain Res. Cogn. Brain Res.

    (2002)
  • SehatpourP. et al.

    Spatiotemporal dynamics of human object recognition processing: an integrated high-density electrical mapping and functional imaging study of “closure” processes

    Neuroimage

    (2006)
  • TrennerM.U. et al.

    Face repetition effects in direct and indirect tasks: an event-related brain potentials study

    Brain Res. Cogn. Brain Res.

    (2004)
  • Anllo-VentoL. et al.

    Selective attention to the color and direction of moving stimuli: electrophysiological correlates of hierarchical feature selection

    Percept. Psychophys.

    (1996)
  • Anllo-VentoL. et al.

    Spatio-temporal dynamics of attention to color: evidence from human electrophysiology

    Hum. Brain Mapp.

    (1998)
  • BentinS. et al.

    The effects of immediare stimulus repetition on reaction time and event-related potentials in tasks of different complexity

    J. Exp. Psychol. Learn. Mem. Cogn.

    (1994)
  • BentinS. et al.

    Electrophysiological studies of face perception in humans

    J. Cogn. Neurosci.

    (1996)
  • BroerseJ. et al.

    The influence of imagery ability on color aftereffects produced by physically present and imagined induction stimuli

    Percept. Psychophys.

    (1980)
  • BroerseJ. et al.

    Investigations of perception and imagery using CAEs: the role of experimental design and psychophysical method

    Percept. Psychophys.

    (1984)
  • D'AngiulliA. et al.

    Generating visual mental images: latency and vividness are inversely related

    Mem. Cognit.

    (2002)
  • D'AngiulliA. et al.

    The relationship between self-reported vividness and latency during mental size scaling of everyday items: phenomenological evidence of different types of imagery

    Am. J. Psychol.

    (2007)
  • DienJ.

    Issues in the application of the average reference: review, critiques, and recommendations

    Behav. Res. Methods, Instrum. Comput.

    (1998)
  • Di RussoF. et al.

    Cortical sources of the early components of the visual evoked potential

    Hum. Brain Mapp.

    (2001)
  • DonigerG.M. et al.

    Activation timecourse of ventral visual stream object-recognition areas: high density electrical mapping of perceptual closure processes

    J. Cogn. Neurosci.

    (2000)
  • FarahM.J. et al.

    Electrophysiological evidence for a shared representational medium for visual images and visual percepts

    J. Exp. Psychol. Gen.

    (1988)
  • FinkeR.A.

    Theories relating mental imagery to perception

    Psychol. Bull.

    (1985)
  • FinkeR.A. et al.

    The quantitative measure of pattern representation in images using orientation-specific color aftereffects

    Percept. Psychophys.

    (1978)
  • GanisG. et al.

    Multiple mechanisms of top-down processing in vision

  • GonsalvesB. et al.

    Neural evidence that vivid imagining can lead to false remembering

    Psychol Sci

    (2004)
  • HarrisA. et al.

    Rapid adaptation of the M170 response: importance of face parts

    Cereb. Cortex.

    (2007)
  • Cited by (62)

    • Beyond imagination: Hypnotic visual hallucination induces greater lateralised brain activity than visual mental imagery

      2021, NeuroImage
      Citation Excerpt :

      See Supplementary Material 2 for a description of the stimulus validation procedure. The task was a modified version of the adaptation paradigm developed by Ganis and Schendan (2008). Prior to the experimental task, while the EEG cap was being applied, participants had to study each stimulus and its corresponding name.

    • Memory influences visual cognition across multiple functional states of interactive cortical dynamics

      2019, Psychology of Learning and Motivation - Advances in Research and Theory
      Citation Excerpt :

      The MUSI account hypothesizes the following. ( 1) Prediction involves mid-VLPFC (BA 47/12; BA 45) and lateral occipital sulcus (LOS) and right fusiform neocortex along the ventral pathway, and this process is indexed by a frontocentral N390 component of the N3 complex (Ganis & Schendan, 2008, 2011; Schendan & Kutas, 2003; Schendan & Stern, 2008). This neurophysiological activity is task-dependent (e.g., involved in categorization more than episodic recognition, though it has a role in both) (Ganis et al., 2007; Jiang et al., 2007; Kosslyn et al., 1994; Raposo, Han, & Dobbins, 2008; Schendan & Stern, 2008).

    View all citing articles on Scopus
    View full text