Controllable Multi-document Summarization: Coverage & Coherence Intuitive Policy with Large Language Model Based Rewards
Memory-efficient large language models are good at refining text input for better readability. However, controllability is a matter of concern when it comes to text generation tasks with long inputs, such as multi-document summarization. In this work, we investigate for a generic controllable approach for multi-document summarization that leverages the capabilities of LLMs to refine the text. In particular, we train a controllable content extraction scheme to extract the text that will be refined by an LLM. The scheme is designed with a novel coverage and coherence intuitive policy, which is duly rewarded by a passively trained LLM. Our approach yields competitive results in the evaluation using ROUGE metrics and outperforms potential baselines in coherence, as per human evaluation.
Code (0)
등록된 구현이 없습니다.
Tasks
Document SummarizationLanguage ModelingLanguage ModellingLarge Language ModelMulti-Document SummarizationText GenerationSimilar Papers 제목 키워드 기반
Leveraging Hierarchical Organization for Medical Multi-document Summarization
Medical multi-document summarization (MDS) is a complex task that requires effectively managing cross-document relationships. This paper investigates whether incorporating hierarchical structures in the inputs of MDS can…
Multi-Document SummarizationMulti-Topic Multi-Document Summarizer
Current multi-document summarization systems can successfully extract summary sentences, however with many limitations including: low coverage, inaccurate extraction to important sentences, redundancy and poor coherence …
Document SummarizationMulti-Document SummarizationDocument Summarization with Conformal Importance Guarantees
Automatic summarization systems have advanced rapidly with large language models (LLMs), yet they still lack reliable guarantees on inclusion of critical content in high-stakes domains like healthcare, law, and finance. …
Document SummarizationLLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction
Multi-document summarization is a challenging task due to its inherent subjective bias, highlighted by the low inter-annotator ROUGE-1 score of 0.4 among DUC-2004 reference summaries. In this work, we aim to enhance the …
Document SummarizationInformativenessLanguage ModelingLanguage Modelling+2FFCI: A Framework for Interpretable Automatic Evaluation of Summarization
In this paper, we propose FFCI, a framework for fine-grained summarization evaluation that comprises four elements: faithfulness (degree of factual consistency with the source), focus (precision of summary content relati…
Question AnsweringSemantic Textual SimilaritySentenceSTS