Audiovisual speech gaze strategies

  • Willy Wong
  • Astrid Yi
  • Moshe Eizenman

Abstract

The goal of this study was to examine the role of gaze in speech perception and to investigate gaze strategies for listening to speech in noise. Eye tracking was conducted on subjects engaged in a noisy audiovisual speech paradigm. Speech intelligibility was measured for eleven subjects listening to low-context sentences while viewing the talking face on a computer monitor. We found that speech intelligibility was similar for all fixations within 10 ÌŠ of the mouth area. However gaze strategy changed with speech signal-to-noise-ratio. When signal-to-noise-ratio was decreased, the number of gaze fixations to mouth region increased as expected. Other experiments were performed whereby gaze was fixed at different eccentricities and speech intelligibility was measured. These results were compared to results which would be obtained by mapping reduced acuity in the peripheral region to various levels of spatial degradation. Our findings suggest that the visual enhancement of speech occurs when subjects are able to see spatial frequencies of 6 cycles/degree or higher.

Section
Full Articles