Adversarial NLI for Factual Correctness in Text Summarisation Models
We apply the Adversarial NLI dataset to train the NLI model and show that the model has the potential to enhance factual correctness in abstract summarization. We follow the work of Falke et al. (2019), which rank multiple generated summaries based on the entailment probabilities between an source document and summaries and select the summary that has the highest entailment probability. The authors' earlier study concluded that current NLI models are not sufficiently accurate for the ranking task. We show that the Transformer models fine-tuned on the new dataset achieve significantly higher accuracy and have the potential of selecting a coherent summary.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisation
Maintaining factual consistency is a critical issue in abstractive text summarisation, however, it cannot be assessed by traditional automatic metrics used for evaluating text summarisation, such as ROUGE scoring. Recent…
The Exploration of Knowledge-Preserving Prompts for Document Summarisation
Despite the great development of document summarisation techniques nowadays, factual inconsistencies between the generated summaries and the original texts still occur from time to time. This study explores the possibili…
Document SummarizationIntegrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour
Autonomous multi-agent systems (MAS) are useful for automating complex tasks but raise trust concerns due to risks like miscoordination and goal misalignment. Explainability is vital for trust calibration, but explainabl…
Autonomous DrivingcounterfactualPredictionEfficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning
Recently, compressive text summarisation offers a balance between the conciseness issue of extractive summarisation and the factual hallucination issue of abstractive summarisation. However, most existing compressive sum…
Hallucinationreinforcement-learningUnsupervised Compressive Text Summarisation with Reinforcement Learning
Recently, compressive text summarisation offers a balance between the conciseness issue of extractive summarisation and the factual hallucination issue of abstractive summarisation. However, most existing compressive sum…
Hallucinationreinforcement-learningReinforcement LearningReinforcement Learning (RL)