SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i.e., quality estimation) assisted by contrastive learning. Experimental results show that, with minor modification over existing top-scoring systems, SimCLS can improve the performance of existing top-performing models by a large margin. Particularly, 2.51 absolute improvement against BART and 2.50 over PEGASUS w.r.t ROUGE-1 on the CNN/DailyMail dataset, driving the state-of-the-art performance to a new level. We have open-sourced our codes and results: https://github.com/yixinL7/SimCLS. Results of our proposed models have been deployed into ExplainaBoard platform, which allows researchers to understand our systems in a more fine-grained way.
Code (2)
Tasks
Abstractive Text SummarizationContrastive LearningText GenerationText SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Factuality of Abstractive Summarization via Contrastive Reward Learning
Modern abstractive summarization models often generate summaries that contain hallucinated or contradictory information. In this paper, we propose a simple but effective contrastive learning framework that incorporates r…
Abstractive Text SummarizationContrastive LearningAlleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization
Most Transformer based abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. From diverse perspectives, we introduce a simple multi-level contrastive learning frame…
Abstractive Text SummarizationContrastive LearningDecoderFew-Shot LearningA Unified and Efficient Contrastive Learning Framework for Text Summarization
Both extractive and abstractive summarization systems share a common problem, i.e., there is a mismatch between the training object and evaluation metrics. To bridge this gap, we introduce a unified and efficient contra…
Abstractive Text SummarizationContrastive LearningText SummarizationContrastive Attention Mechanism for Abstractive Sentence Summarization
We propose a contrastive attention mechanism to extend the sequence-to-sequence framework for abstractive sentence summarization task, which aims to generate a brief summary of a given source sentence. The proposed contr…
Abstractive Text SummarizationSentenceSentence SummarizationText SummarizationEnhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization
In this paper, we present a denoising sequence-to-sequence (seq2seq) autoencoder via contrastive learning for abstractive text summarization. Our model adopts a standard Transformer-based architecture with a multi-layer …
Abstractive Text SummarizationContrastive LearningDecoderDenoising+2