Dialogue System Live Competition: Identifying Problems with Dialogue Systems Through Live Event

Ryuichiro Higashinaka, Kotaro Funakoshi, Michimasa Inaba, Yuiko Tsunomori, Tetsuro Takahashi, Reina Akama

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

We organized a competition entitled “the dialogue system live competition” in which the audience, consisting mainly of researchers in the dialogue community, watched and evaluated a live dialogue conducted between users and dialogue systems. The motivation behind the event was to cultivate state-of-the-art techniques in dialogue systems and enable the dialogue community to share the problems with current dialogue systems. There are two parts to the competition: preliminary selection and live event. In the preliminary selection, eleven systems were evaluated by crowd-sourcing. Three systems proceeded to the live event to perform dialogues with designated speakers and to be evaluated by the audience. This paper describes the design and procedure of the competition, the results of the preliminary selection and live event of the competition, and the problems we identified from the event.

Original languageEnglish
Title of host publicationLecture Notes in Electrical Engineering
PublisherSpringer Science and Business Media Deutschland GmbH
Pages185-199
Number of pages15
DOIs
Publication statusPublished - 2021

Publication series

NameLecture Notes in Electrical Engineering
Volume714
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

ASJC Scopus subject areas

  • Industrial and Manufacturing Engineering

Cite this