ISUAL ENHANCEMENT OF AUDITORY DETECTION: A THEORETICAL MODEL

  • Cheng Qian
  • Willy Wong

Abstract

Audio-visual enhancement is the phenomenon whereby a visual signal can enhance the perception of an auditory signal. This effect has been commonly explored in high-level processes like speech communication yet some aspect of this enhancement can be shown to arise from early sensory processes. Our previous work has shown that the auditory detection threshold of a sinusoidally amplitude modulated tone in quiet is reduced by an average of 2.1 dB when a concurrent, co-modulated visual signal is presented. We report here that the addition of noise does not appear to affect enhancement (average 2.2 dB shift) despite a common notion that noise increases the relative enhancement. We introduce a signal detection model which seeks to quantify the benefit of a co-modulated visual signal. This model is based on the concept of ‘matched filters’ and can account for the improved detection and spread of the psychometric function observed in experiment, as well as shed some light on the roles of synchrony and modulation frequency in cross-modal enhancement.

Section
Full Articles