Multisensory contributions to the perception of motion
Introduction
Our ability to extract information regarding movement is fundamental both for deriving and maintaining useful representations of the environment, and also for the planning and execution of action. As for many other perceptual domains, information regarding the movement of stimuli (i.e. their direction, speed, etc.) is often available via several sensory modalities simultaneously (think, for example, of tracking a partially occluded animal moving through the undergrowth). While vision may provide the most salient information with regard to stimulus motion, audition and somatosensation can also provide important cues, particularly when stimuli are occluded, or else move outside the current field of view (such as when objects move behind the head).
Despite its obvious adaptive importance, and after more than a century of intermittent research on this topic (see Ryan, 1940, Urbantschitsch, 1902; Zietz & Werner, 1927, for early work), our knowledge concerning how motion signals from different sensory modalities are integrated is still fairly limited. Nevertheless, there has been a rapid growth of interest in this area over the last few years, with researchers moving from the phenomenological approaches that characterized many early studies (e.g. Zapparoli & Reatto, 1969) to the development of increasingly sophisticated psychophysical paradigms that are currently enabling researchers to investigate the spatio-temporal constraints on multisensory contributions to the perception of dynamic stimuli.1 Our understanding of these constraints on human perception has also been complemented by recent cognitive neuroscience studies that have begun to reveal the networks of neural structures involved in the integration of motion information across the senses.
In the present review, we start by evaluating the behavioural evidence regarding multisensory contributions to the perception of motion, as well as outlining some of the key constraints affecting such integration. We then proceed to highlight recent neuroimaging and neurophysiological data regarding the neural substrates of motion processing that may underlie some of these multisensory perceptual interactions.2
Section snippets
Multisensory contributions to the perception of motion: behavioral evidence
For many years, researchers have investigated the question of whether the presentation of stimuli in one modality (either moving or stationary) can affect the perception of motion of stimuli presented in another modality (see Soto-Faraco & Kingstone, in press, for a detailed review). Many of these behavioral studies have examined whether the presentation of a static stimulus in one modality affects various aspects of motion processing in another modality, such as its trajectory (e.g. Hall &
Neural correlates of multisensory integration of motion
The brain is rich in both subcortical and cortical areas that demonstrate a sensitivity to various properties of moving stimuli such as directionality or velocity. We now know a great deal about the neural networks involved in visual motion processing, but our knowledge of the networks of brain areas involved in auditory and somatosensory motion processing is currently somewhat more limited. Moreover, the study of how motion information from different modalities interacts in the human brain is
Conclusions
Behavioral research into the nature of crossmodal influences on the perception of stimulus motion has a long, albeit intermittent, history. However, it is only in the last few years, thanks in part to the development of a range of new psychophysical paradigms, that a consistent picture has started to emerge. Recent studies have demonstrated that spatial and temporal factors play an important role in motion integration, just as for many other crossmodal effects. There is now convincing
Acknowledgements
The authors would like to thank E. Macaluso, D. Lloyd, and A. Rodriguez-Fornells for helpful comments on an earlier draft of this manuscript, and also the University of Oxford McDonnell-Pew Centre for Cognitive Neuroscience for funding a Network Grant to S.S.-F. and C.S. Funding to AK was provided by Human Frontier Science Program, the Natural Sciences and Engineering Research Council of Canada, and the Michael Smith Foundation for Health Research. Correspondence regarding this article should
References (139)
Are there neurons detecting direction of sound source motion?
Experimental Neurology
(1968)- et al.
Auditory and visual attention modulate motion processing in area MT+
Cognitive Brain Research
(2002) - et al.
Space coding in primate parietal cortex
Neuroimage
(2001) - et al.
Polymodal motion processing in posterior parietal and premotor cortex: A human fMRI study strongly implies equivalencies between humans and monkeys
Neuron
(2001) - et al.
Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex
Current Biology
(2000) - et al.
Visual areas in the temporal cortex of the macaque
Brain Research
(1979) - et al.
Multisensory perception: Beyond modularity and convergence
Current Biology
(2000) - et al.
A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain
Current Biology
(1996) - et al.
A common cortical substrate activated by horizontal and vertical sound movement in the human brain
Current Biology
(2002) - et al.
Auditory and visual neurons in the cat’s superior colliculus selective for the direction of apparent motion stimuli
Brain Research
(1989)