mFACE: Multilingual Summarization with Factual Consistency Evaluation
Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries, reducing their utility for real-world application. Several recent efforts attempt to address this by devising models that automatically detect factual inconsistencies in machine generated summaries. However, they focus exclusively on English, a language with abundant resources. In this work, we leverage factual consistency evaluation models to improve multilingual summarization. We explore two intuitive approaches to mitigate hallucinations based on the signal provided by a multilingual NLI model, namely data filtering and controlled generation. Experimental results in the 45 languages from the XLSum dataset show gains over strong baselines in both automatic and human evaluation.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationSimilar Papers 제목 키워드 기반
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Reliable automatic evaluation of summarization systems is challenging due to the multifaceted and subjective nature of the task. This is especially the case for languages other than English, where human evaluations are s…
BanglaSummEval: Reference-Free Factual Consistency Evaluation for Bangla Summarization
Evaluating factual consistency is essential for reliable text summarization, particularly in high-stakes domains such as healthcare and news. However, most existing evaluation metrics overlook Bangla, a widely spoken yet…
Question GenerationText SummarizationQuestion AnsweringGradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization
Neural abstractive summarization systems have gained significant progress in recent years. However, abstractive summarization often produce inconsisitent statements or false facts. How to automatically generate highly ab…
Abstractive Text SummarizationData AugmentationImproving Factual Consistency of Abstractive Summarization via Question Answering
A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents. The fact that automatic summarization may pro…
Abstractive Text SummarizationQuestion AnsweringEvaluation of Automatic Text Summarization using Synthetic Facts
Despite some recent advances, automatic text summarization remains unreliable, elusive, and of limited practical use in applications. Two main problems with current summarization methods are well known: evaluation and fa…
Text Summarization