Elsevier

NeuroImage

Volume 48, Issue 1, 15 October 2009, Pages 237-248
NeuroImage

Neural mechanisms of concurrent stimulus processing in dual tasks

https://doi.org/10.1016/j.neuroimage.2009.06.064Get rights and content

Abstract

Little is known about how the human brain processes multiple relevant information streams competing for behavior. The present study aimed at specifying the interaction of the lateral prefrontal cortex (lPFC) with task-relevant sensory brain regions during concurrent stimulus processing of two relevant stimuli (S1, S2) in a classical dual-task situation. In detail, we tested whether S1 processing is independent of the task relevance of S2 as has been hypothesised in cognitive theories on dual-task processing. Using functional Magnetic Resonance Imaging, we tested two neural mechanisms that might reflect effects of S2 relevance on S1 processing at different temporal overlaps. The results indicate that: (1) activity amplitudes in S1-relevant regions in the inferior temporal cortex were similarly affected by the temporal overlap between the two stimuli when S2 was relevant or irrelevant and (2) only when S2 was relevant in the dual task, significant increases in the functional coupling between S1-relevant regions and dual-task-related regions in the posterior lPFC were present at high temporal overlap. No similar effects were found for S2-relevant regions. These findings suggest that concurrent stimulus processing in dual tasks is realised by transient changes in functional coupling for stimuli with relatively higher priority (S1).

Introduction

The limited ability to process multiple information streams simultaneously is one of the most striking characteristics of the human cognitive system. Despite of the highly parallel processing capabilities of the human brain, there seems to exist a fundamental attentional limitation in the processing of temporally overlapping information (Welford, 1952, Pashler, 1994). This limitation is reflected in performance patterns such as the effect of the psychological refractory period (PRP; Welford, 1952, Pashler, 1994) and also other phenomena like the attentional blink (Broadbent and Broadbent, 1987, Marois et al., 2000).

The PRP effect is consistently observed whenever participants respond to two successively presented stimuli (S1 and S2) with pre-defined motor responses. In such a so-called dual-task situation, the response times on S2 (RT2) increase with decreasing temporal overlap between S1 and S2 whereas response times on S1 (RT1) are widely unaffected by the temporal overlap of the two stimuli. Most cognitive models explain the PRP effect with the assumption that certain processing stages related to S2 processing are interrupted when there is temporal overlap with the corresponding processing stages in the S1 processing chain (Welford, 1952, Pashler, 1994). This interference between S1 and S2 processing leads to the prolongation of the processing time for S2 and thus provides an explanation for the PRP effect.

Although the neural mechanisms of the PRP effect recently became a prominent research topic (Dux et al., 2006, Schubert and Szameitat, 2003, Stelzel et al., 2008), little is known about the involved brain dynamics related to the processing of interference between two sensory stimuli in a PRP situation. Until now, most PRP neuroimaging studies focused on isolating specific brain regions related to the PRP effect or to the coordination of the interfering processing streams in the PRP paradigm. As a prominent region, the lateral prefrontal cortex (lPFC) has been identified to be crucially involved in PRP processing (Dux et al., 2006; Jiang, 2004; Schubert and Szameitat, 2003, Stelzel et al., 2006, Szameitat et al., 2002). For example, Dux et al. (2006) showed with rapid event-related fMRI that the posterior lPFC (plPFC) is critically involved in the processing delay of S2 in a PRP situation (see also Marois and Ivanoff, 2005). In our own studies, we also found this region in the posterior section of the inferior frontal sulcus (IFS) to be involved in the control of task coordination in PRP-like dual-task situations (Stelzel et al., 2008, Szameitat et al., 2002, Szameitat et al., 2006).

While most dual-task studies focused on the isolated functioning of the lPFC during PRP processing, recent evidence suggests that the lPFC is cooperatively involved in larger brain networks whenever task processes need to be coordinated in interference situations. For the case of single-task interference, a network of different regions has been shown to be relevant for managing the evolving interference, including the lPFC, posterior sensory brain regions (Egner and Hirsch, 2005, Yeung et al., 2006), medial prefrontal regions (Botvinick et al., 2004, Kerns et al., 2004) and further cortical regions. Only recently, Egner and Hirsch (2005) used fMRI to demonstrate this in a variant of the Stroop task. They showed that not only the activity changes in the lPFC but, additionally, activity changes in posterior brain regions and, importantly, the modulatory influence of the first on the latter are important for understanding the whole complexity of interference processing (see also Yeung et al., 2006). In their study, participants categorized visually presented faces, overlayed by the names of actors or politicians, into faces of actors and faces of politicians. Congruent conditions were those in which the category of the face stimulus coincided with the category of the written name as compared to incongruent conditions where both categories contradicted each other. The authors found an amplification of the brain activity in task-relevant sensory regions in the fusiform gyrus in so-called high-control compared to low-control situations. High vs. low control refers to effects of conflict adaptation in the Stroop paradigm (see also Kerns et al., 2004) where recent Stroop trials were either preceded by an incongruent Stroop trial (high control) or a congruent Stroop trial (low control). Importantly, signal amplification in task-relevant sensory brain regions was associated with reduced behavioral interference effects as measured by the comparison of the RTs in incongruent and congruent Stroop-conditions in the recent trial. These effects were further accompanied by a modulatory influence of the lPFC as revealed by a connectivity analysis (see below). Thus, in single-task situations increased activity in task-relevant sensory regions seems to be critically associated with reduced interference from irrelevant sensory inputs that compete for the same motor behavior as it is the case in incongruent Stroop-like situations.

Unlike the Stroop task, the PRP task represents a dual-task situation, in which two sensory inputs, S1 and S2, are both relevant for subsequent behavior within the same task trial. This renders the task situation more complex than a single-task interference situation because S2 must also be processed to a degree so that the second task can still be performed correctly, albeit delayed. It is not known yet whether in such situations of two concurrently relevant stimuli, similar modulations of activity in task-relevant sensory brain regions occur as in the respective conditions of interference with irrelevant stimulus information in single tasks (Egner and Hirsch, 2005). Classical models on information processing in PRP situations (Pashler, 1994) postulate S1 processing to be independent of the presence of a secondary task. This, in particular, would predict that brain activity related to S1 processing should not be affected by the relevance of a simultaneously presented stimulus S2. Some recent findings, however, indicate that S1 processing may be impaired under certain conditions of temporal overlap between S1 and S2 (Hein and Schubert, 2004, Hein et al., 2005, Jolicœur and Dell'Acqua, 1999). In particular, effects on S1 processing have been found when S2 is highly salient and thus interferes with S1 on a perceptual level, leading to increased processing demands for S1 as reflected in increases in RT1.

Critically, the pure use of RT measures may not be sufficient to obtain the whole dynamics of possible perceptual interference between concurrent processing of S1 and S2 (De Jong and Sweet, 1994, Pashler and Johnston, 1998). The analysis of differential changes in the blood-oxygenation-level-dependent (BOLD) signals in task-relevant brain regions for S1 and/or S2 processing might represent an additional tool to detect processing demands related to concurrent stimulus processing in dual tasks (Jiang, 2004). Therefore, the first aim of the present study was to test whether the concurrent processing of two visual stimuli in a dual-task situation of the PRP type is associated with a modulation of the activity in S1-relevant regions and whether this depends on the degree of the temporal overlap with S2.

In addition to the analysis of modulations in the activity of posterior brain regions, we also tested for modulatory effects of the lPFC on the activity in posterior regions. In particular, we tested whether differences in the temporal overlap of S1 and S2 are related to differences in the functional coupling between sensory regions relevant for S1 and S2 processing and dual-task-related regions in the lPFC. Miller and Cohen (2001) defined the central role of the lPFC to be the biasing of signals to other brain regions in order to guide the flow of activity along neural pathways in accordance with internal goals. Consistent with this assumption, Egner and Hirsch (2005) used functional connectivity measures in their study to measure the mutual influence of the neural signals in the lPFC and sensory task-relevant regions applying the method of psychophysiological interactions (PPI; Friston et al., 1997). As mentioned earlier, they showed that the functional coupling between the lPFC and the task-relevant sensory brain regions was increased in high-control situations allowing for efficient interference processing as compared to low-control situations. Egner and Hirsch (2005) interpreted this finding with the assumption that efficient functional coupling of sensory regions with the lPFC is an important neural mechanism for resolving interference between competing processing streams.

Applied to the processing of dual-task interference, efficient S1 processing at high temporal overlap with S2 may be accompanied by increased functional coupling of S1-relevant sensory regions with dual-task-related regions in the plPFC. By strengthening the functional coupling of the S1-relevant sensory regions with the plPFC, transient priority may be given to S1 processing when there is high temporal overlap with S2 processing, thus protecting S1 processing from interference with S2. For S2-relevant regions, such a modulation of the functional coupling with the plPFC depending on the temporal overlap is hypothesised to be less strong than for S1-relevant regions. This is because the increase of RT2 with increasing temporal overlap with S1 suggests an interruption of the S2-related processing chain, which may indicate that the corresponding S2 processing stream is less protected from interference with S1 than vice versa.

To elucidate the involvement of these two neural mechanisms, e.g. the modulation of activity amplitudes and/or functional coupling, in concurrent stimulus processing in dual tasks, we measured the neural activity of participants during the performance of a PRP task consisting of a face discrimination (S1) and a number comparison (S2) task. The face stimulus was always presented first. The number stimulus was presented with varying stimulus onset asynchronies (SOAs) after S1. We applied a regions-of-interest approach, first determining the sensory brain regions related to the sensory processing of S1 (here: Fusiform Face Area (FFA)) and S2 (here: Visual Word Form Area (VWFA)) in independent localizer tasks (Kanwisher et al., 1997, Fiez and Petersen, 1998, Cohen et al., 2002, Dehaene et al., 2002). In addition, regions in the lPFC related to processes of dual-task coordination (Stelzel et al., 2008, Szameitat et al., 2002; for details see Methods) were determined with a separate localizer task. In the main PRP experiment, participants performed a manual choice reaction depending on the gender of a presented face stimulus (male/female; S1) and a choice reaction depending on the size of a number word (</> five; S2), giving priority to the face task. In order to disentangle the effects of concurrent stimulus processing from effects of overlapping stimulus presentation, we additionally manipulated the task relevance of S2 — while S2 was irrelevant for task performance in single-task blocks, in dual-task blocks, S2 also required a manual response.

We measured the activity changes in S1-relevant regions in the FFA, S2-relevant regions in the VWFA and dual-task-related regions in the plPFC. In addition, we used functional connectivity analysis to determine the changes in the functional coupling between the sensory regions and the plPFC depending on the temporal overlap. Importantly, effects of temporal overlap in dual tasks on the measures of neural activity and of functional coupling were expected for the S1-relevant regions in the FFA, but not for the S2-relevant regions in the VWFA.

Section snippets

Participants

Fourteen healthy right-handed volunteers with normal or corrected to normal vision participated in the experiment (eight males, ages 20–30, mean age: 24.9, SDV: 3.1) after obtaining informed consent according to the Declaration of Helsinki. Participants were paid 10 € per hour.

Main experiment

Participants performed a face and a number task in different blocks of single-task and dual-task trials in a mixed block and event-related fMRI design (see Fig. 1). In every trial, a black and white face stimulus (FERET

Behavioral results

Mean reaction times (RTs) for all task conditions are shown in Fig. 2, separately for the FACE task (RT1) and the NUMBER task (RT2). For the FACE task, we conducted a two-factorial ANOVA with repeated measures (factors S2 RELEVANCE and SOA). We found an effect of S2 RELEVANCE, F(1, 13) = 18.37, MSE = 3877.11, p < .001, with increased RT1 in the DUAL TASK (mean [m] = 670.8 ms) compared to the SINGLE TASK (m = 612.6 ms). In addition, we found an effect of SOA, F(2, 26) = 10.56, MSE = 621.22. p < .001, and a

Discussion

Studies on the neural basis of dual-task processing in PRP tasks often focused primarily on the functional role of the lPFC (e.g., Dux et al., 2006, Schubert and Szameitat, 2003). The present investigation goes beyond this by investigating the activity changes in task-relevant sensory brain regions during dual-task processing and the interaction of these brain regions with dual-task-related regions in the lPFC. In particular, we asked whether the temporal overlap between two visual stimuli S1

Acknowledgments

This work is part of the PhD of C.S. and was supervised by T.S. The work was supported by a grant of the Deutsche Forschungsgemeinschaft to T.S. (Schu-1397/2-3) and by a grant of the Sonnenfeld-Stiftung to C.S. We are grateful to Grit Herzmann for help with the stimulus material and to Stefanie Kehrer and Antje Kraft for help in data acquisition.

References (55)

  • MaroisR. et al.

    Neural correlates of the attentional blink

    Neuron

    (2000)
  • MaroisR. et al.

    The neural fate of consciously perceived and missed events in the attentional blink

    Neuron

    (2004)
  • NirY. et al.

    Coupling between neuronal firing rate, gamma LFP, and BOLD fMRI is related to interneuronal correlations

    Curr. Biol.

    (2007)
  • PhillipsP.J. et al.

    FERET database and evaluation procedure for face-recognition algorithms

    Image Vis. Comput.

    (1998)
  • SchubertT. et al.

    Functional neuroanatomy of interference in overlapping dual tasks: an fMRI study

    Brain Res. Cogn. Brain Res.

    (2003)
  • BauerM. et al.

    Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas

    J. Neurosci.

    (2006)
  • BroadbentD.E. et al.

    From detection to identification: response to multiple targets in rapid serial visual presentation

    Percept. Psychophys.

    (1987)
  • ChaoL.L. et al.

    Human prefrontal lesions increase distractibility to irrelevant sensory inputs

    NeuroReport

    (1995)
  • CohenL. et al.

    Language-specific tuning of visual cortex? Functional properties of the visual word form area

    Brain

    (2002)
  • de FockertJ.W. et al.

    The role of working memory in visual selective attention

    Science

    (2001)
  • DehaeneS. et al.

    The visual word form area: a prelexical representation of visual words in the fusiform gyrus

    NeuroReport

    (2002)
  • De JongR.

    The role of preparation in overlapping-task performance

    Q. J. Exp. Psychol., A

    (1995)
  • De JongR. et al.

    Preparatory strategies in overlapping-task performance

    Percept. Psychophys.

    (1994)
  • D'EspositoM. et al.

    The neural basis of the central executive system of working memory

    Nature

    (1995)
  • EgnerT. et al.

    Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information

    Nat. Neurosci.

    (2005)
  • FiezJ.A. et al.

    Neuroimaging studies of word reading

    Proc. Natl. Acad. Sci. U. S. A.

    (1998)
  • FristonK.J. et al.

    Statistical parametric maps in functional imaging: a general linear approach

    Hum. Brain Mapp.

    (1995)
  • Cited by (61)

    • Modality-specific effects of mental fatigue in multitasking

      2022, Acta Psychologica
      Citation Excerpt :

      Dreisbach and Wenke (2011) found that shielding is active during task repetitions, presumably protecting the actual performance from interference produced by the other, at that moment irrelevant, task set, and loose during task switches. In the context of dual-task processing, prioritization of task one in a PRP paradigm has been associated with similar mechanisms (Sigman & Dehaene, 2006; Stelzel et al., 2009). Inferring from the initially high dual-task costs for the modality incompatible tasks compared to the modality compatible tasks, participants may not allocate enough effort to shield the relevant task mapping.

    • Dual-tasking or concurrent multitasking

      2021, Encyclopedia of Behavioral Neuroscience: Second Edition
    View all citing articles on Scopus
    View full text