To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text
Abstractive neural summarization models have seen great improvements in recent years, as shown by ROUGE scores of the generated summaries. But despite these improved metrics, there is limited understanding of the strategies different models employ, and how those strategies relate their understanding of language. To understand this better, we run several experiments to characterize how one popular abstractive model, the pointer-generator model of See et al. (2017), uses its explicit copy/generation switch to control its level of abstraction (generation) vs extraction (copying). On an extractive-biased dataset, the model utilizes syntactic boundaries to truncate sentences that are otherwise often copied verbatim. When we modify the copy/generation switch and force the model to generate, only simple paraphrasing abilities are revealed alongside factual inaccuracies and hallucinations. On an abstractive-biased dataset, the model copies infrequently but shows similarly limited abstractive abilities. In line with previous research, these results suggest that abstractive summarization models lack the semantic understanding necessary to generate paraphrases that are both abstractive and faithful to the source document.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationSimilar Papers 제목 키워드 기반
Understanding Points of Correspondence between Sentences for Abstractive Summarization
Fusing sentences containing disparate content is a remarkable human ability that helps create informative and succinct summaries. Such a simple task for humans has remained challenging for modern abstractive summarizers,…
Abstractive Text Summarizationcoreference-resolutionCoreference ResolutionSentence+1Assessing LLMs for Zero-shot Abstractive Summarization Through the Lens of Relevance Paraphrasing
Large Language Models (LLMs) have achieved state-of-the-art performance at zero-shot generation of abstractive summaries for given articles. However, little is known about the robustness of such a process of zero-shot su…
Abstractive Text SummarizationArticlesUnderstanding the Behaviour of Neural Abstractive Summarizers using Contrastive Examples
Neural abstractive summarizers generate summary texts using a language model conditioned on the input source text, and have recently achieved high ROUGE scores on benchmark summarization datasets. We investigate how they…
Language ModelingLanguage ModellingAnalyzing Sentence Fusion in Abstractive Summarization
While recent work in abstractive summarization has resulted in higher scores in automatic metrics, there is little understanding on how these systems combine information taken from multiple document sentences. In this pa…
Abstractive Text SummarizationSentenceSentence FusionSemantic Extractor-Paraphraser based Abstractive Summarization
The anthology of spoken languages today is inundated with textual information, necessitating the development of automatic summarization models. In this manuscript, we propose an extractor-paraphraser based abstractive su…
Abstractive Text Summarization