Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also proposed a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on the Gigaword headline generation and DUC2004 datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform length-transfer summary generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Headline GenerationSentenceSentence SummarizationText SummarizationUnsupervised Sentence SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS…
SentenceSentence SummarizationText SummarizationUnsupervised Sentence SummarizationDiscrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction
Automatic sentence summarization produces a shorter version of a sentence, while preserving its most important information. A good summary is characterized by language fluency and high information overlap with the source…
Language ModelingLanguage ModellingSemantic SimilaritySemantic Textual Similarity+3BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
The principle of the Information Bottleneck (Tishby et al. 1999) is to produce a summary of information X optimized to predict some other relevant information Y. In this paper, we propose a novel approach to unsupervised…
Abstractive Text SummarizationExtractive SummarizationLanguage ModelingLanguage Modelling+4A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization
Sentence summarization aims at compressing a long sentence into a short one that keeps the main gist, and has extensive real-world applications such as headline generation. In previous work, researchers have developed va…
Headline GenerationSentenceSentence SummarizationImproving Sentence Similarity Estimation for Unsupervised Extractive Summarization
Unsupervised extractive summarization aims to extract salient sentences from a document as the summary without labeled data. Recent literatures mostly research how to leverage sentence similarity to rank sentences in the…
Contrastive LearningExtractive SummarizationSentenceSentence Similarity+1