Effects of Spatial Congruity on Audio-Visual Multimodal Integration W. A. Teder-Sa ¨leja ¨rvi 1 , F. Di Russo 2 , J. J. McDonald 3 , and S. A. Hillyard 1 Abstract & Spatial constraints on multisensory integration of audi- tory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded re- sponses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bi- modal stimuli were faster and more accurate than to the uni- modal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural inter- actions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spa- tial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100–400 msec and an amplitude modulation of activity localized to the superior temporal region at 260– 280 msec. These results demonstrate overlapping but distinc- tive patterns of multisensory integration for spatially congruent and incongruent AV stimuli. & INTRODUCTION Many objects and events in the natural world have multimodal features that stimulate different sensory pathways. Despite the initial segregation of the senses, information about the different features of an object is integrated in the brain to form unified multisensory percepts. Studies of human performance have revealed numerous behavioral consequences of multisensory integration. Under many circumstances, for example, the simultaneous stimulation of two or more modalities has a facilitatory effect on behavioral performance, with bimodal stimuli being detected and discriminated faster and more accurately than the constituent unimodal stimuli presented alone (for reviews, see Frassinetti, Bolognini, & La `davas, 2002; Frens, Van Opstal, & Van der Willigen, 1995; Welsh & Warren, 1986; Loveless, Brebner, & Hamilton, 1970). In groundbreaking studies of the neural bases of multisensory integration, Stein (1998) and Stein and Meredith (1993) discovered neurons in the cat brain that were responsive to more than one modality. These multisensory neurons were found in the superior collic- ulus as well as in certain cortical association areas. Many of these cells were particularly responsive to concurrent bimodal stimulation when the constituent stimuli ap- peared in close spatial proximity. For such cells, the neu- ral response to a bimodal stimulus was larger than the sum of the responses to the individual unimodal stimuli. This multisensory response enhancement parallels the aforementioned behavioral facilitation resulting from bimodal stimulation and was proposed as the neural basis for the integration of unimodal sensations into unified multimodal percepts (Stein & Meredith, 1993). The neural bases of multisensory integration have been studied in humans using neuroimaging techniques (Beauchamp, Lee, Argall, & Martin, 2004; Foxe, Wylie, et al., 2002; Laurienti et al., 2002; Calvert, Hansen, Iversen, & Brammer, 2001; Calvert, Campbell, & Bram- mer, 2000) and recordings of event-related brain poten- tials (ERPs) (Fort, Delpuech, Pernier, & Giard, 2002; Molholm et al., 2002; Teder-Sa ¨leja ¨rvi, McDonald, Di Russo, & Hillyard, 2002; Foxe, Morocz, et al., 2000; Giard & Peronnet, 1999; Schro ¨ger & Widmann, 1998) and magnetic fields (Raij, Uutela, & Hari, 2000). In several of these ERP studies, auditory–visual integration was investigated by comparing the ERP to a bimodal audio- visual (AV) stimulus with the sum of the ERPs to the constituent auditory (A) and visual (V) stimuli. Multi- sensory interactions were revealed in the difference 1 University of California—San Diego, 2 University Institute of Motor Sciences (IUSM) and Santa Lucia Foundation IRCCS, Rome, Italy, 3 Simon Fraser University, Vancouver, Canada D 2005 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 17:9, pp. 1396–1409