TrueBrief: Faithful Summarization through Small Language Models
Large language models (LLMs) have exhibited remarkable proficiency in generating high-quality text; however, their propensity for producing hallucinations poses a significant challenge for their deployment in security-critical domains. In this work, we present TrueBrief, an end-to-end framework specifically designed to enhance the faithfulness of small LLMs (SLMs) primarily for the task of text summarization through a preference-optimization paradigm. Central to our framework is a data generation module that facilitates controlled hallucination injection to generate synthetic preference data. Our work provides insights into the impact of data quality and model size on preference-based optimization, highlighting the conditions under which these methods are most effective.
Code (0)
등록된 구현이 없습니다.
Tasks
Text SummarizationSimilar Papers 제목 키워드 기반
FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization
Summaries of medical text shall be faithful by being consistent and factual with source inputs, which is an important but understudied topic for safety and efficiency in healthcare. In this paper, we investigate and impr…
Contrastive LearningTowards Improving Faithfulness in Abstractive Summarization
Despite the success achieved in neural abstractive summarization based on pre-trained language models, one unresolved issue is that the generated summaries are not always faithful to the input document. There are two pos…
Abstractive Text SummarizationDecoderLanguage ModelingLanguage Modelling+1Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization
Recent pre-trained language models (PLMs) achieve promising results in existing abstractive summarization datasets. However, existing summarization benchmarks overlap in time with the standard pre-training corpora and fi…
Abstractive Text SummarizationText SummarizationED-FAITH: Evaluating Dialogue Summarization on Faithfulness
Abstractive summarization models typically generate content unfaithful to the input, thus highlighting the significance of evaluating the faithfulness of generated summaries. Most faithfulness metrics are only evaluated …
Abstractive Text SummarizationLanguage ModelingLanguage ModellingDon't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models
Large Language Models (LLMs) are adept at text manipulation -- tasks such as machine translation and text summarization. However, these models can also be prone to hallucination, which can be detrimental to the faithfuln…
HallucinationMachine TranslationText SummarizationTranslation