Effect of speed difference between time-expanded speech and moving image of talker's face on word intelligibility

Shuichi Sakamoto, Akihiro Tanaka, Komi Tsumura, Yôiti Suzuki

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

This study investigated effects of asynchrony between speech signal and moving image of talker's face induced by time-expansion of the speech signal on speech intelligibility. Word intelligibility test was performed to younger listeners. Japanese 4-mora words were uttered by a female speaker. Each word was processed with STRAIGHT software to expand the speech signal by from 0 to 400 ms. These signals were combined with moving image of talker's face which was kept at original speed. This test was performed under three conditions: visual-only, auditory-only, and auditory-visual (AV) condition. Results showed that intelligibility scores under AV condition were statistically higher than those under auditory-only condition even when the speech signal was expanded by 400 ms. These results suggest that moving image of talker's face is effective to enhance speech intelligibility if the lag between the speech signal and moving image of talker's face does not exceed 400 ms.

Original languageEnglish
Pages (from-to)199-203
Number of pages5
JournalJournal on Multimodal User Interfaces
Volume2
Issue number3
DOIs
Publication statusPublished - 2008 Dec

Keywords

  • Audio-visual interaction
  • Lip-reading
  • Moving image of talker's face
  • Time-expanded speech
  • Word intelligibillity

ASJC Scopus subject areas

  • Signal Processing
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Effect of speed difference between time-expanded speech and moving image of talker's face on word intelligibility'. Together they form a unique fingerprint.

Cite this