PromptSum: Parameter-Efficient Controllable Abstractive Summarization
Prompt tuning (PT), a parameter-efficient technique that only tunes the additional prompt embeddings while keeping the backbone pre-trained language model (PLM) frozen, has shown promising results in language understanding tasks, especially in low-resource scenarios. However, effective prompt design methods suitable for generation tasks such as summarization are still lacking. At the same time, summarization guided through instructions (discrete prompts) can achieve a desirable double objective of high quality and controllability in summary generation. Towards a goal of strong summarization performance under the triple conditions of parameter-efficiency, data-efficiency, and controllability, we introduce PromptSum, a method combining PT with a multi-task objective and discrete entity prompts for abstractive summarization. Our model achieves competitive ROUGE results on popular abstractive summarization benchmarks coupled with a strong level of controllability through entities, all while only tuning several orders of magnitude less parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Length-controllable Abstractive Summarization by Guiding with Summary Prototype
We propose a new length-controllable abstractive summarization model. Recent state-of-the-art abstractive summarization models based on encoder-decoder models generate only one summary per source text. However, controlla…
Abstractive Text SummarizationDecoderMACSum: Controllable Summarization with Mixed Attributes
Controllable summarization allows users to generate customized summaries with specified attributes. However, due to the lack of designated annotations of controlled summaries, existing works have to craft pseudo datasets…
ArticlesAttributeSpecificityControllable Topic-Focused Abstractive Summarization
Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects by shifting the distribution of generated text towards a desired style, e.g., a set of topics. Su…
Abstractive Text SummarizationInterpretable Multi-Headed Attention for Abstractive Summarization at Controllable Lengths
Abstractive summarization at controllable lengths is a challenging task in natural language processing. It is even more challenging for domains where limited training data is available or scenarios in which the length of…
Abstractive Text SummarizationTransfer LearningTopic-controllable Abstractive Summarization
Existing approaches for topic-controllable summarization either incorporate topic embeddings or modify the attention mechanism. The incorporation of such approaches in a particular summarization model requires the adapta…
Abstractive Text Summarization