Skip to main content
Log in

How the bimodal format of presentation affects working memory: an overview

  • Review
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

The best format in which information that has to be recalled is presented has been investigated in several studies, which focused on the impact of bimodal stimulation on working memory performance. An enhancement of participant’s performance in terms of correct recall has been repeatedly found, when bimodal formats of presentation (i.e., audiovisual) were compared to unimodal formats (i.e, either visual or auditory), in providing implications for multimedial learning. Several theoretical frameworks have been suggested in order to account for the bimodal advantage, ranging from those emphasizing early stages of processing (such as automatic alerting effects or multisensory integration processes) to those centred on late stages of processing (as postulated by the dual coding theory). The aim of this paper is to review previous contributions to this topic, providing a comprehensive theoretical framework, which is updated by the latest empirical studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Baddeley A (1990) Human memory: theory and practice. Lawrence Erlbaum Associates Inc, Hillsdale

    Google Scholar 

  • Baddeley A (1992) Working memory. Science 255:556–559

    Article  PubMed  CAS  Google Scholar 

  • Baddeley AD (2000) The episodic buffer: a new component of working memory? Trends Cogn Sci 4:417–423

    Article  PubMed  Google Scholar 

  • Baddeley AD, Hitch GJ (1974) Working memory. In: Bower G (ed) The psychology of learning and motivation. Academic, New York, pp 47–89

    Google Scholar 

  • Bauer MI, Johnson-Laird PN (1993) How diagrams can improve reasoning. Psychol Sci 6:372–378

    Article  Google Scholar 

  • Buckingham HW (1981) Lexical and semantic aspects of aphasia. In: Sarno MT (ed) Acquired aphasia. Academic, New York, pp 183–214

    Google Scholar 

  • Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649–657

    Article  PubMed  CAS  Google Scholar 

  • Campbell R, Dodd B (1980) Hearing by eye. Q J Exp Psychol 32:85–99

    Article  PubMed  CAS  Google Scholar 

  • Crowder RG, Morton J (1969) Precategorical acoustic storage (PAS). Percept Psychophys 5:363–373

    Google Scholar 

  • De Gelder B, Vroomen J (1992) Abstract versus modality-specific memory representations. Mem Cognit 20:533–538

    PubMed  Google Scholar 

  • De Gelder B, Vroomen J (1994) Memory for consonants versus vowels in heard and lipread speech. J Mem Lang 31:737–756

    Article  Google Scholar 

  • De Gelder B, Vroomen J (1997) Modality effects in immediate recall of verbal and non-verbal information. Eur J Cogn Psychol 9:97–110

    Article  Google Scholar 

  • Engelkamp J, Zimmer HD (1994) The human memory: a multi-modal approach. Hogrefe, Toronto

    Google Scholar 

  • Foos PW, Goolkasian P (2005) Presentation format effects in working memory: the role of attention. Mem Cognit 33:499–513

    PubMed  Google Scholar 

  • Fort A, Delpuech C, Pernier J, Giard MH (2002) Early auditory–visual interactions in human cortex during nonredundant target identification. Cogn Brain Res 14:20–30

    Article  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory–visual integration during multimodal object recognition in humans: a behavioural and electrophysiological study. J Cogn Neurosci 11:473–490

    Article  PubMed  CAS  Google Scholar 

  • Goodglass H (1980) Disorders of naming following brain injury. Am Sci 68:647–655

    PubMed  CAS  Google Scholar 

  • Goolkasian H (1996) Picture-word difference in a sentence verification task. Mem Cognit 24:584–594

    PubMed  CAS  Google Scholar 

  • Goolkasian P (2000) Pictures, words, and sounds: from which format are we best able to reason? J Gen Psychol 127:439–459

    Article  PubMed  CAS  Google Scholar 

  • Goolkasian P, Foos PW (2002) Presentation format and its effect on working memory. Mem Cognit 30:1096–1105

    PubMed  Google Scholar 

  • Goolkasian P, Foos PW (2005) Bimodal format effects in working memory. Am J Psychol 118:61–77

    PubMed  Google Scholar 

  • Greene RL, Samuel AG (1986) Recency and suffix effects in serial recall of musical stimuli. J Exp Psychol Learn Mem Cogn 12:517–524

    Article  PubMed  CAS  Google Scholar 

  • Hasher L, Zacks RT (1979) Automatic and effortful processes in memory. J Exp Psychol Gen 108:356–388

    Article  Google Scholar 

  • Johnson CJ, Paivio A, Clark JM (1996) Cognitive components of picture naming. Psychol Bull 120:113–139

    Article  PubMed  CAS  Google Scholar 

  • Jones DM (1994) Disruption of memory for lip read lists by irrelevant speech: further support for the changing state hypothesis. Q J Exp Psychol 47A:143–160

    Google Scholar 

  • Jones DM, Macken WJ (1993) Irrelevant tones produce an irrelevant speech effect: implications for phonological coding in working memory. J Exp Psychol Learn Mem Cogn 19:369–381

    Article  Google Scholar 

  • Jones DM, Macken WJ (1995) Phonological similarity in the irrelevant speech effect: within- or between-stream similarity? J Exp Psychol Learn Mem Cogn 21:103–133

    Article  Google Scholar 

  • Larkin JH, Simon HA (1987) Why an illustration is (sometimes) worth ten thousand words. Cogn Sci 11:65–99

    Article  Google Scholar 

  • Mayer RE, Sims VK (1994) Why a diagram is (sometimes) worth ten thousand words. Cogn Sci 11:65–69

    Google Scholar 

  • Morton J (1969) Interaction of information in word recognition. Psychol Rev 76:165–178

    Article  Google Scholar 

  • Morton J (1979) Facilitation in word recognition: experiments causing change in the logogen model. In: Kolers PA, Wrolstead M, Bouma H (eds) Processing of visible language, vol 1. Plenum Press, New York, pp 259–268

    Google Scholar 

  • Oberauer K (2002) Access to information in working memory: exploring the focus of attention. J Exp Psychol Learn Mem Cogn 28:411–421

    Article  PubMed  Google Scholar 

  • Oberauer K, Demmruch A, Mayr U, Kilegl R (2001) Dissociating retention and access in working memory: an age-comparative study of mental arithmetic. Mem Cognit 29:18–33

    PubMed  CAS  Google Scholar 

  • Olivetti Belardinelli M, Sestieri C, Di Matteo R, Delogu F, Del Gratta C, Ferretti A, Caulo M, Tartaro M, Romani GL (2004) Audio–visual crossmodal interactions in environmental perception: An fMRI investigation. Cogn Process 5:167–174

    Google Scholar 

  • Paivio A (1971) Imagery and verbal processes. Holt, Rinehart and Winston, New York

    Google Scholar 

  • Paivio A (1986) Mental representations: a dual coding approach. Oxford University Press, Oxford

    Google Scholar 

  • Penney CG (1975) Modality effects in short-term verbal memory. Psychol Bull 82:68–84

    Article  Google Scholar 

  • Penney CG (1989) Modality effects and the structure of short-term verbal memory. Mem Cognit 17:398–422

    PubMed  CAS  Google Scholar 

  • Santangelo V, Spence C (2007) Multisensory cues capture spatial attention regardless of perceptual load. J Exp Psychol Hum Percept Perform (in press)

  • Santangelo V, Mastroberardino S, Botta F, Marucci FS, Olivetti Belardinelli M (2006) On the influence of audio–visual interactions on working memory performance: a study with non-semantic stimuli. Cogn Process 7:187

    Article  Google Scholar 

  • Santangelo V, Olivetti Belardinelli M, Spence C (2007) The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. J Exp Psychol Hum Percept Perform 33:137–148

    Article  PubMed  Google Scholar 

  • Spoehr KT, Corin WJ (1978) The stimulus suffix effect as a memory coding phenomenon. Mem Cognit 6:583–589

    PubMed  CAS  Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge

    Google Scholar 

  • Surprenant AM, LeCompte DC, Neath I (2000) Manipulations of irrelevant information: suffix effects with articulatory suppression and irrelevant speech. Q J Exp Psychol A 53:325–348

    Article  PubMed  CAS  Google Scholar 

  • Sweller J, Chandler P, Tierney P, Cooper M (1990) Cognitive load as a factor in the structuring of technical material. J Exp Psychol Gen 119:176–192

    Article  Google Scholar 

  • Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002) An analysis of audio–visual crossmodal integration by means of event-related potential (ERP) recordings. Cogn Brain Res 14:106–114

    Article  Google Scholar 

  • Thompson V, Paivio A (1994) Memory for pictures and sounds: independence of auditory and visual codes. Can J Exp Psychol 48:380–396

    Article  PubMed  CAS  Google Scholar 

  • Tindall-Ford S, Chandler P, Sweller J (1997) When two sensory modes are better than one. J Exp Psychol Appl 3:257–287

    Article  Google Scholar 

Download references

Acknowledgment

We wish to thank Angela Tagini for the language revision.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Serena Mastroberardino.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mastroberardino, S., Santangelo, V., Botta, F. et al. How the bimodal format of presentation affects working memory: an overview. Cogn Process 9, 69–76 (2008). https://doi.org/10.1007/s10339-007-0195-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-007-0195-6

Keywords

Navigation