paper-with-me

홈 › Papers

mFACE: Multilingual Summarization with Factual Consistency Evaluation

2022-12-20 · Roee Aharoni, Shashi Narayan, Joshua Maynez, Jonathan Herzig, Elizabeth Clark, Mirella Lapata

Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries, reducing their utility for real-world application. Several recent efforts attempt to address this by devising models that automatically detect factual inconsistencies in machine generated summaries. However, they focus exclusively on English, a language with abundant resources. In this work, we leverage factual consistency evaluation models to improve multilingual summarization. We explore two intuitive approaches to mitigate hallucinations based on the signal provided by a multilingual NLI model, namely data filtering and controlled generation. Experimental results in the 45 languages from the XLSum dataset show gains over strong baselines in both automatic and human evaluation.

📄 PDF Abstract BibTeX arXiv:2212.10622

Code (0)

등록된 구현이 없습니다.

Tasks

Abstractive Text Summarization

Similar Papers 제목 키워드 기반

SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation

2023-05-22 · Elizabeth Clark, Shruti Rijhwani, Sebastian Gehrmann, Joshua Maynez 외

Reliable automatic evaluation of summarization systems is challenging due to the multifaceted and subjective nature of the task. This is especially the case for languages other than English, where human evaluations are s…

BanglaSummEval: Reference-Free Factual Consistency Evaluation for Bangla Summarization

2026-02-18 · Ahmed Rafid, Rumman Adib, Fariya Ahmed, Ajwad Abrar 외 arxiv

Evaluating factual consistency is essential for reliable text summarization, particularly in high-stakes domains such as healthcare and news. However, most existing evaluation metrics overlook Bangla, a widely spoken yet…

Question GenerationText SummarizationQuestion Answering

Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization

2021-11-01 · EMNLP 2021 11 · Zhiyuan Zeng, Jiaze Chen, Weiran Xu, Lei LI

Neural abstractive summarization systems have gained significant progress in recent years. However, abstractive summarization often produce inconsisitent statements or false facts. How to automatically generate highly ab…

Abstractive Text SummarizationData Augmentation

Improving Factual Consistency of Abstractive Summarization via Question Answering

2021-05-10 · ACL 2021 5 · Feng Nan, Cicero Nogueira dos santos, Henghui Zhu, Patrick Ng 외

A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents. The fact that automatic summarization may pro…

Abstractive Text SummarizationQuestion Answering

Evaluation of Automatic Text Summarization using Synthetic Facts

2022-04-11 · Jay Ahn, Foaad Khosmood

Despite some recent advances, automatic text summarization remains unreliable, elusive, and of limited practical use in applications. Two main problems with current summarization methods are well known: evaluation and fa…

Text Summarization