Unsupervised language model adaptation based on automatic text collection from WWW

Motoyuki Suzuki, Yasutomo Kajiura, Akinori Ito, Shozo Makino

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)

Abstract

An n-gram trained by a general corpus gives high performance. However, it is well known that a topic-specialized n-gram gives higher performance than that of the general n-gram. In order to make a topic specialized n-gram, several adaptation methods were proposed. These methods use a given corpus corresponding to the target topic, or collect documents related to the topic from a database. If there is neither the given corpus nor the topic-related documents in the database, the general n-gram cannot be adapted to the topic-specialized n-gram. In this paper, a new unsupervised adaptation method is proposed. The method collects topic-related documents from the world wide web. Several query terms are extracted from recognized text, and collected web pages given by a search engine are used for adaptation. Experimental results showed the proposed method gave 7.2 points higher word accuracy than that given by the general n-gram.

Original languageEnglish
Title of host publicationINTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP
PublisherInternational Speech Communication Association
Pages2202-2205
Number of pages4
ISBN (Print)9781604234497
Publication statusPublished - 2006 Jan 1
EventINTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP - Pittsburgh, PA, United States
Duration: 2006 Sep 172006 Sep 21

Publication series

NameINTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP
Volume5

Other

OtherINTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP
CountryUnited States
CityPittsburgh, PA
Period06/9/1706/9/21

Keywords

  • Google
  • Language model adaptation
  • Quary-based sampling
  • Search engine
  • World wide web

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Unsupervised language model adaptation based on automatic text collection from WWW'. Together they form a unique fingerprint.

  • Cite this

    Suzuki, M., Kajiura, Y., Ito, A., & Makino, S. (2006). Unsupervised language model adaptation based on automatic text collection from WWW. In INTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP (pp. 2202-2205). (INTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP; Vol. 5). International Speech Communication Association.