Hallucination Reduction in Long Input Text Summarization
Hallucination in text summarization refers to the phenomenon where the model generates information that is not supported by the input source document. Hallucination poses significant obstacles to the accuracy and reliability of the generated summaries. In this paper, we aim to reduce hallucinated outputs or hallucinations in summaries of long-form text documents. We have used the PubMed dataset, which contains long scientific research documents and their abstracts. We have incorporated the techniques of data filtering and joint entity and summary generation (JAENS) in the fine-tuning of the Longformer Encoder-Decoder (LED) model to minimize hallucinations and thereby improve the quality of the generated summary. We have used the following metrics to measure factual consistency at the entity level: precision-source, and F1-target. Our experiments show that the fine-tuned LED model performs well in generating the paper abstract. Data filtering techniques based on some preprocessing steps reduce entity-level hallucinations in the generated summaries in terms of some of the factual consistency metrics.
Code (1)
Tasks
DecoderHallucinationText SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Tackling Hallucinations in Neural Chart Summarization
Hallucinations in text generation occur when the system produces text that is not grounded in the input. In this work, we tackle the problem of hallucinations in neural chart summarization. Our analysis shows that the ta…
Data-to-Text GenerationTable-to-Text GenerationText GenerationHallucinate at the Last in Long Response Generation: A Case Study on Long Document Summarization
Large Language Models (LLMs) have significantly advanced text generation capabilities, including tasks like summarization, often producing coherent and fluent outputs. However, faithfulness to source material remains a s…
Document SummarizationHallucinationResponse GenerationText GenerationChallenges in Domain-Specific Abstractive Summarization and How to Overcome them
Large Language Models work quite well with general-purpose data and many tasks in Natural Language Processing. However, they show several limitations when used for a task such as domain-specific abstractive text summariz…
Abstractive Text SummarizationHallucinationText SummarizationGradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation
In tasks like summarization and open-book question answering (QA), Large Language Models (LLMs) often encounter "contextual hallucination", where they produce irrelevant or incorrect responses despite having access to ac…
Computational EfficiencyHallucinationQuestion AnsweringImproved Beam Search for Hallucination Mitigation in Abstractive Summarization
Advancement in large pretrained language models has significantly improved their performance for conditional language generation tasks including summarization albeit with hallucinations. To reduce hallucinations, convent…
Abstractive Text SummarizationDiversityHallucinationNatural Language Inference+2