SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We propose an end-to-end neural model for zero-shot abstractive text summarization of paragraphs, and introduce a benchmark task, ROCSumm, based on ROCStories, a subset for which we collected human summaries. In this task, five-sentence stories (paragraphs) are summarized with one sentence, using human summaries only for evaluation. We show results for extractive and human baselines to demonstrate a large abstractive gap in performance. Our model, SummAE, consists of a denoising auto-encoder that embeds sentences and paragraphs in a common space, from which either can be decoded. Summaries for paragraphs are generated by decoding a sentence from the paragraph representations. We find that traditional sequence-to-sequence auto-encoders fail to produce good summaries and describe how specific architectural choices and pre-training techniques can significantly improve performance, outperforming extractive baselines. The data, training, evaluation code, and best model weights are open-sourced.
Code (2)
Tasks
Abstractive Text SummarizationDenoisingSentenceText SummarizationSimilar Papers 제목 키워드 기반
Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias
We characterize and study zero-shot abstractive summarization in Large Language Models (LLMs) by measuring position bias, which we propose as a general formulation of the more restrictive lead bias phenomenon studied pre…
Abstractive Text SummarizationDecoderPositionFew-shot fine-tuning SOTA summarization models for medical dialogues
Abstractive summarization of medical dialogues presents a challenge for standard training approaches, given the paucity of suitable datasets. We explore the performance of state-of-the-art models with zero-shot and few-s…
Abstractive Text SummarizationFew-Shot LearningRestructuring Conversations using Discourse Relations for Zero-shot Abstractive Dialogue Summarization
Dialogue summarization is a challenging problem due to the informal and unstructured nature of conversational data. Recent advances in abstractive summarization have been focused on data-hungry neural models and adapting…
Abstractive Dialogue SummarizationAbstractive Text SummarizationDocument SummarizationImproving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets. However, such models have been shown to be more prone to hallucinate facts that are unfaithf…
Abstractive Text SummarizationImproving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets. However, such models have been shown to be more prone to hallucinate facts that are unfaithf…
Abstractive Text Summarization