Psychological Science
22(7) 872–877
© The Author(s) 2011
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0956797611413293
http://pss.sagepub.com
Most physical events generate signals that can be encoded in
multiple sensory modalities, such as audition and vision.
Because of the vast difference between the speeds of light and
sound, visual and auditory signals are subject to distance-
dependent changes in the relative times at which they reach
people’s sensory receptors (see Alais & Carlile, 2005; Arnold,
Johnston, & Nishida, 2005; King, 2005; Spence & Squire,
2003; Sugita & Suzuki, 2003). On the basis of their intensity,
sensory signals are also subject to changes in the rates at which
they propagate through the central nervous system (Burr &
Corsale, 2001; Lennie, 1981). However, at least for proximate
stimuli, humans are seldom aware of any asynchrony between
an events’ multisensory components. One reason for this might
be that the brain has some capacity to correct for extrinsically
and intrinsically generated differences in sensory coding
times. One plausible mechanism for this function is audiovi-
sual temporal recalibration. Following prolonged exposure
(adaptation) to streams of asynchronous auditory and visual
events, timing perception can recalibrate, such that the timing
of the adapted asynchronous signals seems more synchronous
than it did previously (Di Luca, Machulla, & Ernst, 2009;
Fujisaki, Shimojo, Kashino, & Nishida, 2004; Hanson, Heron,
& Whitaker, 2008; Harrar & Harris, 2008; Heron, Whitaker,
McGraw, & Horoshenkov, 2007; Keetels & Vroomen, 2007;
Miyazaki, Yamamoto, Uchida, & Kitazawa, 2006; Navarra
et al., 2005; Navarra, Hartcher-O’Brien, Piazza, & Spence,
2009; Vatakis, Navarra, Soto-Faraco, & Spence, 2007, 2008;
Vroomen, Keetels, de Gelder, & Bertleson, 2004). This sub-
jective recalibration of audiovisual synchrony is often accom-
panied by increases in the range of timing differences across
which auditory and visual signals seem synchronous (Hanson,
Heron, & Whitaker, 2008; Navarra et al., 2005; Vatakis et al.,
2007, 2008).
Although recalibration of audiovisual timing perception
could be beneficial, humans exist in a cluttered environment
and can concurrently encounter many events that generate
multisensory signals. Given that the signals generated by these
events may be subject to different delays due to variable view-
ing distances and signal intensities, it might not be beneficial
Corresponding Author:
Warrick Roseboom, University of Queensland, School of Psychology,
McElwain Building, St. Lucia, Brisbane, Queensland 4072, Australia
E-mail: w.roseboom@psy.uq.edu.au
Twice Upon a Time: Multiple Concurrent
Temporal Recalibrations of Audiovisual
Speech
Warrick Roseboom and Derek H. Arnold
The University of Queensland
Abstract
Audiovisual timing perception can recalibrate following prolonged exposure to asynchronous auditory and visual inputs. It has
been suggested that this might contribute to achieving perceptual synchrony for auditory and visual signals despite differences
in physical and neural signal times for sight and sound. However, given that people can be concurrently exposed to multiple
audiovisual stimuli with variable neural signal times, a mechanism that recalibrates all audiovisual timing percepts to a
single timing relationship could be dysfunctional. In the experiments reported here, we showed that audiovisual temporal
recalibration can be specific for particular audiovisual pairings. Participants were shown alternating movies of male and
female actors containing positive and negative temporal asynchronies between the auditory and visual streams. We found that
audiovisual synchrony estimates for each actor were shifted toward the preceding audiovisual timing relationship for that actor
and that such temporal recalibrations occurred in positive and negative directions concurrently. Our results show that humans
can form multiple concurrent estimates of appropriate timing for audiovisual synchrony.
Keywords
audiovisual simultaneity, audiovisual recalibration, subjective synchrony, audiovisual timing, temporal recalibration, audiovisual
grouping, cross-modal grouping
Received 9/28/10; Revision accepted 3/16/11
Research Report