Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document. We improve these important aspects of abstractive summarization via multi-task learning with the auxiliary tasks of question generation and entailment generation, where the former teaches the summarization model how to look for salient questioning-worthy details, and the latter teaches the model how to rewrite a summary which is a directed-logical subset of the input document. We also propose novel multi-task architectures with high-level (semantic) layer-specific sharing across multiple encoder and decoder layers of the three tasks, as well as soft-sharing mechanisms (and show performance ablations and analysis examples of each contribution). Overall, we achieve statistically significant improvements over the state-of-the-art on both the CNN/DailyMail and Gigaword datasets, as well as on the DUC-2002 transfer setup. We also present several quantitative and qualitative analysis studies of our model's learned saliency and entailment skills.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationDecoderMulti-Task LearningQuestion GenerationQuestion-GenerationSimilar Papers 제목 키워드 기반
EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization
Code summarization is the task of generating natural language descriptions of source code, which is critical for software comprehension and maintenance. While large language models (LLMs) have achieved remarkable progres…
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond
Is the output softmax layer, which is adopted by most language models (LMs), always the best way to compute the next word probability? Given so many attention layers in a modern transformer-based LM, are the pointer netw…
Analysis on LLMs Performance for Code Summarization
Code summarization aims to generate concise natural language descriptions for source code. Deep learning has been used more and more recently in software engineering, particularly for tasks like code creation and summari…
Code SummarizationTowards Visually Grounded Multimodal Summarization via Cross-Modal Transformer and Gated Attention
Multimodal summarization requires models to jointly understand textual and visual inputs to generate concise, semantically coherent summaries. Existing methods often inject shallow visual features into deep language mode…
Text SummarizationPoint ProcessesExtractive Multi-document Summarization Using Multilayer Networks
Huge volumes of textual information has been produced every single day. In order to organize and understand such large datasets, in recent years, summarization techniques have become popular. These techniques aims at fin…
Document SummarizationMulti-Document Summarization