Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Hearing visual motion in depth

Nature volume 416, pages 172–174 (2002)Cite this article

Abstract

Auditory spatial perception is strongly affected by visual cues. For example, if auditory and visual stimuli are presented synchronously but from different positions, the auditory event is mislocated towards the locus of the visual stimulus—the ventriloquism effect1,2. This ‘visual capture’ also occurs in motion perception in which a static auditory stimulus appears to move with the visual moving object3,4. We investigated how the human perceptual system coordinates complementary inputs from auditory and visual senses. Here we show that an auditory aftereffect occurs from adaptation to visual motion in depth. After a few minutes of viewing a square moving in depth, a steady sound was perceived as changing loudness in the opposite direction. Adaptation to a combination of auditory and visual stimuli changing in a compatible direction increased the aftereffect and the effect of visual adaptation almost disappeared when the directions were opposite. On the other hand, listening to a sound changing in intensity did not affect the visual changing-size aftereffect. The results provide psychophysical evidence that, for processing of motion in depth, the auditory system responds to both auditory changing intensity and visual motion in depth.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Magnitude of the auditory changing-loudness aftereffect after adaptation to combinations of auditory changing-intensity and visual changing-size stimuli.
Figure 2: Schematic illustration of approaching visual stimulus.
Figure 3: Effect of visual adaptation to changing-disparity on the auditory changing-loudness aftereffect.
Figure 4: The visual changing-size aftereffect for four combinations in expanding and shrinking directions.

Similar content being viewed by others

References

  1. Jack, C. E. & Thurlow, W. R. Effects of degree of visual association and angle of displacement on the “ventriloquism” effect. Percept. Motor Skill. 37, 967–979 (1973).

    CAS  Google Scholar 

  2. Bertelson, P. & Radeau, M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29, 578–584 (1981).

    Article  CAS  Google Scholar 

  3. Mateeff, S., Hohnsbein, J. & Noack, T. Dynamic visual capture: apparent auditory motion induced by a moving visual target. Perception 14, 721–727 (1985).

    Article  CAS  Google Scholar 

  4. Kitajima, N. & Yamashita, Y. Dynamic capture of sound motion by light stimuli moving in three-dimensional space. Percept. Motor Skill. 89, 1139–1158 (1999).

    Article  CAS  Google Scholar 

  5. Wade, N. J. A selective history of the study of visual motion aftereffects. Perception 23, 1111–1134 (1994).

    Article  CAS  Google Scholar 

  6. Mather, G., Verstraten, F. & Anstis, S. The Motion Aftereffect: A Modern Perspective (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  7. Grantham, D. W. Motion aftereffects with horizontally moving sound sources in the free field. Percept. Psychophys. 45, 129–136 (1989).

    Article  CAS  Google Scholar 

  8. Dong, C. J., Swindale, N. V., Zakarauskas, P., Hayward, V. & Cynader, M. S. The auditory motion aftereffect: its tuning and specificity in the spatial and frequency domains. Percept. Psychophys. 62, 1099–1111 (2000).

    Article  CAS  Google Scholar 

  9. Shu, Z. J., Swindale, N. V. & Cynader, M. S. Spectral motion produces an auditory after-effect. Nature 364, 721–723 (1993).

    Article  ADS  CAS  Google Scholar 

  10. Reinhardt-Rutland, A. H. & Anstis, S. Auditory adaptation to gradual rise or fall in intensity of a tone. Percept. Psychophys. 31, 63–67 (1982).

    Article  CAS  Google Scholar 

  11. Reinhardt-Rutland, A. H. Increasing-loudness aftereffect following decreasing-intensity adaptation: spectral dependence in interotic and monotic testing. Perception 27, 473–482 (1998).

    Article  CAS  Google Scholar 

  12. Kayahara, T. Aftereffect of adaptation to uni-direction frequency change: evidence for selective processing mechanism. Acoust. Sci. Technol. 22, 49–51 (2001).

    Article  Google Scholar 

  13. Regan, D. & Beverley, K. I. Illusory motion in depth: aftereffect of adaptation to changing size. Vision Res. 18, 209–212 (1978).

    Article  CAS  Google Scholar 

  14. Cumming, B. in Visual Detection of Motion (eds Smith, A. T. & Snowden, R. J.) 333–366 (Academic, London, 1994).

    Google Scholar 

  15. Ehrenstein, W. H. & Reinhardt-Rutland, A. H. A cross-modal aftereffect: auditory displacement following adaptation to visual motion. Percept. Motor Skill. 82, 23–26 (1996).

    Article  CAS  Google Scholar 

  16. Welch, R. B. & Warren, D. H. in Handbook of Perception and Human Performance. Vol. 1 Sensory Processes and Perception (eds Boff, K. R., Kaufman, L. & Thomas, J. P.) Ch. 25 (Wiley, New York, 1986).

    Google Scholar 

  17. Blauert, J. Spatial Hearing: the Psychophysics of Human Sound Localization Revised edn (MIT Press, Cambridge, Massachusetts, 1997).

    Google Scholar 

  18. Calvert, G. A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997).

    Article  CAS  Google Scholar 

  19. Macaluso, E., Frith, C. D. & Driver, J. Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208 (2000).

    Article  ADS  CAS  Google Scholar 

  20. Lewis, J. W., Beauchamp, M. S. & DeYoe, E. A. A comparison of visual and auditory motion processing in human cerebral cortex. Cereb. Cortex 10, 873–888 (2000).

    Article  CAS  Google Scholar 

  21. Driver, J. & Spence, C. Multisensory perception: beyond modularity and convergence. Curr. Biol. 10, R731–R735 (2000).

    Article  CAS  Google Scholar 

  22. Calvert, G. A., Campbell, R. & Brammer, M. J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10, 649–657 (2000).

    Article  CAS  Google Scholar 

  23. Brainard, D. H. The psychophysics toolbox. Spatial Vision 10, 433–436 (1997).

    Article  CAS  Google Scholar 

  24. Pelli, D. G. The Video Toolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision 10, 437–442 (1997).

    Article  CAS  Google Scholar 

  25. Cornsweet, T. N. The staircase-method in psychophysics. Am. J. Psychol. 75, 485–491 (1962).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank D. Levi, H. Akutsu and D. Erickson for helpful comments and suggestions.

Author information

Author notes

  1. Norimichi Kitagawa: Correspondence and requests for materials should be addressed to N.K.

Authors and Affiliations

  1. Department of Psychology, Tokyo Metropolitan University, 1-1, Minami-Osawa, Hachioji-shi, 192-0397, Tokyo, Japan

    Norimichi Kitagawa & Shigeru Ichihara

Authors
  1. Norimichi Kitagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigeru Ichihara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Norimichi Kitagawa.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kitagawa, N., Ichihara, S. Hearing visual motion in depth. Nature 416, 172–174 (2002). https://doi.org/10.1038/416172a

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/416172a

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing