An unsupervised language model adaptation based on keyword clustering and query availability estimation

Akinori Ito, Yasutomo Kajiura, Shozo Makino, Motoyuki Suzuki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Language model adaptation using text data downloaded from the WWW is an efficient way to train a topic-specific LM. We are developing an unsupervised LM adaptation method using data in the Web. The one key point of unsupervised Web-based LM adaptation is how to select keywords to compose the search query. In this paper, we propose a new method of selecting keywords from keyword candidates, which uses a keyword clustering technique based on word similarities. The other key point is how to determine the number of downloaded pages for each query. In this paper we propose a method to estimate "a query availability," which is based on a small number of downloaded Web pages. The experimental result showed that the determination of downloaded pages using the query availability was effective than the conventional methods that determined the number of pages empirically.

Original languageEnglish
Title of host publicationICALIP 2008 - 2008 International Conference on Audio, Language and Image Processing, Proceedings
Pages1412-1418
Number of pages7
DOIs
Publication statusPublished - 2008 Sep 22
EventICALIP 2008 - 2008 International Conference on Audio, Language and Image Processing - Shanghai, China
Duration: 2008 Jul 72008 Jul 9

Publication series

NameICALIP 2008 - 2008 International Conference on Audio, Language and Image Processing, Proceedings

Other

OtherICALIP 2008 - 2008 International Conference on Audio, Language and Image Processing
CountryChina
CityShanghai
Period08/7/708/7/9

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'An unsupervised language model adaptation based on keyword clustering and query availability estimation'. Together they form a unique fingerprint.

Cite this