Pyramid-based Summary Evaluation Using Abstract Meaning Representation
We propose a novel metric for evaluating summary content coverage. The evaluation framework follows the Pyramid approach to measure how many summarization content units, considered important by human annotators, are contained in an automatic summary. Our approach automatizes the evaluation process, which does not need any manual intervention on the evaluated summary side. Our approach compares abstract meaning representations of each content unit mention and each summary sentence. We found that the proposed metric complements well the widely-used ROUGE metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstract Meaning RepresentationGraph SimilaritySentenceSimilar Papers 제목 키워드 기반
Text Summarization using Abstract Meaning Representation
With an ever increasing size of text present on the Internet, automatic summary generation remains an important problem for natural language understanding. In this work we explore a novel full-fledged pipeline for text s…
Abstract Meaning RepresentationNatural Language UnderstandingText SummarizationOn the Role of Summary Content Units in Text Summarization Evaluation
At the heart of the Pyramid evaluation method for text summarization lie human written summary content units (SCUs). These SCUs are concise sentences that decompose a summary into small facts. Such SCUs can be used to ju…
Natural Language InferenceSentenceText SummarizationAbstract Meaning Representation for Multi-Document Summarization
Generating an abstract from a collection of documents is a desirable capability for many real-world applications. However, abstractive approaches to multi-document summarization have not been thoroughly investigated. Thi…
Abstract Meaning RepresentationDocument SummarizationMulti-Document SummarizationJoint Parsing and Generation for Abstractive Summarization
Sentences produced by abstractive summarization systems can be ungrammatical and fail to preserve the original meanings, despite being locally fluent. In this paper we propose to remedy this problem by jointly generating…
Abstractive Text SummarizationDecoderSentenceText SummarizationTR-EduVSum: A Turkish-Focused Dataset and Consensus Framework for Educational Video Summarization
This study presents a framework for generating the gold-standard summary fully automatically and reproducibly based on multiple human summaries of Turkish educational videos. Within the scope of the study, a new dataset …
Video Summarization