An annotation protocol for collecting user-generated counter-arguments using crowdsourcing

Paul Reisert, Gisela Vallejo, Naoya Inoue, Iryna Gurevych, Kentaro Inui

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Constructive feedback is important for improving critical thinking skills. However, little work has been done to automatically generate such feedback for an argument. In this work, we experiment with an annotation protocol for collecting user-generated counter-arguments via crowdsourcing. We conduct two parallel crowdsourcing experiments, where workers are instructed to produce (i) a counter-argument, and (ii) a counter-argument after identifying a fallacy. Our analysis indicates that we can collect counter-arguments that are useful as constructive feedback, especially when workers are first asked to identify a fallacy type.

Original languageEnglish
Title of host publicationArtificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings
EditorsSeiji Isotani, Eva Millán, Amy Ogan, Bruce McLaren, Peter Hastings, Rose Luckin
PublisherSpringer Verlag
Pages232-236
Number of pages5
ISBN (Print)9783030232061
DOIs
Publication statusPublished - 2019
Event20th International Conference on Artificial Intelligence in Education, AIED 2019 - Chicago, United States
Duration: 2019 Jun 252019 Jun 29

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11626 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Artificial Intelligence in Education, AIED 2019
Country/TerritoryUnited States
CityChicago
Period19/6/2519/6/29

Keywords

  • Annotation study
  • Constructive feedback
  • Counter-argument
  • Critical thinking
  • Crowdsourcing
  • Fallacy

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'An annotation protocol for collecting user-generated counter-arguments using crowdsourcing'. Together they form a unique fingerprint.

Cite this