Encoder-decoder Network as Loss Function for Summarization
We present a new approach to defining a sequence loss function to train a summarizer by using a secondary encoder-decoder as a loss function, alleviating a shortcoming of word level training for sequence outputs. The technique is based on the intuition that if a summary is a good one, it should contain the most essential information from the original article, and therefore should itself be a good input sequence, in lieu of the original, from which a summary can be generated. We present experimental results where we apply this additional loss function to a general abstractive summarizer on a news summarization dataset. The result is an improvement in the ROUGE metric and an especially large improvement in human evaluations, suggesting enhanced performance that is competitive with specialized state-of-the-art models.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderNews SummarizationSimilar Papers 제목 키워드 기반
Video Summarization with Attention-Based Encoder-Decoder Networks
This paper addresses the problem of supervised video summarization by formulating it as a sequence-to-sequence learning problem, where the input is a sequence of original video frames, the output is a keyshot sequence. O…
DecoderSupervised Video SummarizationVideo SummarizationGypSum: Learning Hybrid Representations for Code Summarization
Code summarization with deep learning has been widely studied in recent years. Current deep learning models for code summarization generally follow the principle in neural machine translation and adopt the encoder-decode…
Code SummarizationDecoderDeep LearningGraph Attention+3Weakly-supervised Video Summarization using Variational Encoder-Decoder and Web Prior
Video summarization is a challenging under-constrained problem because the underlying summary of a single video strongly depends on users' subjective understandings. Data-driven approaches, such as deep neural networks, …
DecoderSaliency PredictionSupervised Video SummarizationVideo SummarizationImproving the Factual Accuracy of Abstractive Clinical Text Summarization using Multi-Objective Optimization
While there has been recent progress in abstractive summarization as applied to different domains including news articles, scientific articles, and blog posts, the application of these techniques to clinical text summari…
Abstractive Text SummarizationArticlesDecoderText SummarizationSparsity and Sentence Structure in Encoder-Decoder Attention of Summarization Systems
Transformer models have achieved state-of-the-art results in a wide range of NLP tasks including summarization. Training and inference using large transformer models can be computationally expensive. Previous work has fo…
Abstractive Text SummarizationDecoderDocument SummarizationSentence