Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries

Benjamin Heinzerling, Kentaro Inui

研究成果: Conference contribution

11 被引用数 (Scopus)

抄録

Pretrained language models have been suggested as a possible alternative or complement to structured knowledge bases. However, this emerging LM-as-KB paradigm has so far only been considered in a very limited setting, which only allows handling 21k entities whose name is found in common LM vocabularies. Furthermore, a major benefit of this paradigm, i.e., querying the KB using natural language paraphrases, is underexplored. Here we formulate two basic requirements for treating LMs as KBs: (i) the ability to store a large number facts involving a large number of entities and (ii) the ability to query stored facts. We explore three entity representations that allow LMs to handle millions of entities and present a detailed case study on paraphrased querying of facts stored in LMs, thereby providing a proof-of-concept that language models can indeed serve as knowledge bases.

本文言語English
ホスト出版物のタイトルEACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference
出版社Association for Computational Linguistics (ACL)
ページ1772-1791
ページ数20
ISBN(電子版)9781954085022
出版ステータスPublished - 2021
イベント16th Conference of the European Chapter of the Associationfor Computational Linguistics, EACL 2021 - Virtual, Online
継続期間: 2021 4月 192021 4月 23

出版物シリーズ

名前EACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference

Conference

Conference16th Conference of the European Chapter of the Associationfor Computational Linguistics, EACL 2021
CityVirtual, Online
Period21/4/1921/4/23

ASJC Scopus subject areas

  • ソフトウェア
  • 計算理論と計算数学
  • 言語学および言語

フィンガープリント

「Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル