Named entity recognition with multiple segment representations

Han Cheol Cho, Naoaki Okazaki, Makoto Miwa, Jun'Ichi Tsujii

Research output: Contribution to journalArticlepeer-review

29 Citations (Scopus)


Named entity recognition (NER) is mostly formalized as a sequence labeling problem in which segments of named entities are represented by label sequences. Although a considerable effort has been made to investigate sophisticated features that encode textual characteristics of named entities (e.g. PEOPLE, LOCATION, etc.), little attention has been paid to segment representations (SRs) for multi-token named entities (e.g. the IOB2 notation). In this paper, we investigate the effects of different SRs on NER tasks, and propose a feature generation method using multiple SRs. The proposed method allows a model to exploit not only highly discriminative features of complex SRs but also robust features of simple SRs against the data sparseness problem. Since it incorporates different SRs as feature functions of Conditional Random Fields (CRFs), we can use the well-established procedure for training. In addition, the tagging speed of a model integrating multiple SRs can be accelerated equivalent to that of a model using only the most complex SR of the integrated model. Experimental results demonstrate that incorporating multiple SRs into a single model improves the performance and the stability of NER. We also provide the detailed analysis of the results.

Original languageEnglish
Pages (from-to)954-965
Number of pages12
JournalInformation Processing and Management
Issue number4
Publication statusPublished - 2013


  • Conditional random fields
  • Feature engineering
  • Machine learning
  • Named entity recognition

ASJC Scopus subject areas

  • Information Systems
  • Media Technology
  • Computer Science Applications
  • Management Science and Operations Research
  • Library and Information Sciences


Dive into the research topics of 'Named entity recognition with multiple segment representations'. Together they form a unique fingerprint.

Cite this