Crossmodal Enhancement: Investigating the role of timing in audio-visual enhancement

  • Sheena Luu
  • Willy Wong

Abstract

The results of a set of three psychophysical experiments investigating crossmodal enhancement using sinusoidally amplitude modulated (SAM) audio and visual stimuli are presented. The experiments are designed to probe the importance of timing in crossmodal enhancement. Our findings suggest that synchrony is a requisite for enhancement to occur. The results also suggest that although some enhancement may be due to the visual stimuli providing timing information for the detection of the audio stimuli, timing information alone cannot explain the full magnitude of the observed enhancement. Given the importance of synchrony perception in crossmodal integration, a model of audio-visual synchrony detection based on windowed cross-correlation is also introduced and discussed.

Section
Full Articles