Enumeration of extractive oracle summaries

Tsutomu Hirao; Masaaki Nishino; Jun Suzuki; Masaaki Nagata

doi:10.18653/v1/e17-1037

Enumeration of extractive oracle summaries

Tsutomu Hirao, Masaaki Nishino, Jun Suzuki, Masaaki Nagata

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Citations (Scopus)

Abstract

To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGEn. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

Original language	English
Title of host publication	Long Papers
Publisher	Association for Computational Linguistics (ACL)
Pages	386-396
Number of pages	11
ISBN (Electronic)	9781510838604
DOIs	https://doi.org/10.18653/v1/e17-1037
Publication status	Published - 2017
Externally published	Yes
Event	15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Valencia, Spain Duration: 2017 Apr 3 → 2017 Apr 7

Publication series

Name	15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference
Volume	1

Other

Other	15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017
Country/Territory	Spain
City	Valencia
Period	17/4/3 → 17/4/7

ASJC Scopus subject areas

Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/e17-1037

Cite this

Enumeration of extractive oracle summaries. / Hirao, Tsutomu; Nishino, Masaaki; Suzuki, Jun et al.

Long Papers. Association for Computational Linguistics (ACL), 2017. p. 386-396 (15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hirao, T, Nishino, M, Suzuki, J & Nagata, M 2017, Enumeration of extractive oracle summaries. in Long Papers. 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference, vol. 1, Association for Computational Linguistics (ACL), pp. 386-396, 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, 17/4/3. https://doi.org/10.18653/v1/e17-1037

@inproceedings{51d7a6d89956414288a7a6b83980d405,

title = "Enumeration of extractive oracle summaries",

abstract = "To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGEn. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.",

author = "Tsutomu Hirao and Masaaki Nishino and Jun Suzuki and Masaaki Nagata",

note = "Publisher Copyright: {\textcopyright} 2017 Association for Computational Linguistics.; 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 ; Conference date: 03-04-2017 Through 07-04-2017",

year = "2017",

doi = "10.18653/v1/e17-1037",

language = "English",

series = "15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "386--396",

booktitle = "Long Papers",

}

TY - GEN

T1 - Enumeration of extractive oracle summaries

AU - Hirao, Tsutomu

AU - Nishino, Masaaki

AU - Suzuki, Jun

AU - Nagata, Masaaki

PY - 2017

Y1 - 2017

N2 - To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGEn. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

AB - To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGEn. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

UR - http://www.scopus.com/inward/record.url?scp=85021694017&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021694017&partnerID=8YFLogxK

U2 - 10.18653/v1/e17-1037

DO - 10.18653/v1/e17-1037

M3 - Conference contribution

AN - SCOPUS:85021694017

T3 - 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference

SP - 386

EP - 396

BT - Long Papers

PB - Association for Computational Linguistics (ACL)

T2 - 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017

Y2 - 3 April 2017 through 7 April 2017

ER -

Enumeration of extractive oracle summaries

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this