A unified learning framework of skip-grams and global vectors

Jun Suzuki, Masaaki Nagata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Log-bilinear language models such as SkipGram and GloVe have been proven to capture high quality syntactic and seman-tic relationships between words in a vector space. We revisit the relationship between SkipGram and GloVe models from a ma-chine learning viewpoint, and show that these two methods are easily merged into a unified form. Then, by using the unified form, we extract the factors of the config-urations that they use differently. We also empirically investigate which factor is re-sponsible for the performance difference often observed in widely examined word similarity and analogy tasks.

Original languageEnglish
Title of host publicationACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages186-191
Number of pages6
ISBN (Electronic)9781941643730
Publication statusPublished - 2015 Jan 1
Externally publishedYes
Event53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL-IJCNLP 2015 - Beijing, China
Duration: 2015 Jul 262015 Jul 31

Publication series

NameACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference
Volume2

Other

Other53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL-IJCNLP 2015
CountryChina
CityBeijing
Period15/7/2615/7/31

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Language and Linguistics
  • Linguistics and Language

Fingerprint Dive into the research topics of 'A unified learning framework of skip-grams and global vectors'. Together they form a unique fingerprint.

  • Cite this

    Suzuki, J., & Nagata, M. (2015). A unified learning framework of skip-grams and global vectors. In ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference (pp. 186-191). (ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference; Vol. 2). Association for Computational Linguistics (ACL).