Dialogue System Live Competition: Identifying Problems with Dialogue Systems Through Live Event

Ryuichiro Higashinaka, Kotaro Funakoshi, Michimasa Inaba, Yuiko Tsunomori, Tetsuro Takahashi, Reina Akama

    Research output: Chapter in Book/Report/Conference proceedingChapter


    We organized a competition entitled “the dialogue system live competition” in which the audience, consisting mainly of researchers in the dialogue community, watched and evaluated a live dialogue conducted between users and dialogue systems. The motivation behind the event was to cultivate state-of-the-art techniques in dialogue systems and enable the dialogue community to share the problems with current dialogue systems. There are two parts to the competition: preliminary selection and live event. In the preliminary selection, eleven systems were evaluated by crowd-sourcing. Three systems proceeded to the live event to perform dialogues with designated speakers and to be evaluated by the audience. This paper describes the design and procedure of the competition, the results of the preliminary selection and live event of the competition, and the problems we identified from the event.

    Original languageEnglish
    Title of host publicationLecture Notes in Electrical Engineering
    PublisherSpringer Science and Business Media Deutschland GmbH
    Number of pages15
    Publication statusPublished - 2021

    Publication series

    NameLecture Notes in Electrical Engineering
    ISSN (Print)1876-1100
    ISSN (Electronic)1876-1119

    ASJC Scopus subject areas

    • Industrial and Manufacturing Engineering

    Cite this