WiCE: Real-World Entailment for Claims in Wikipedia
Textual entailment models are increasingly applied in settings like fact-checking, presupposition verification in question answering, or summary evaluation. However, these represent a significant domain shift from existing entailment datasets, and models underperform as a result. We propose WiCE, a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia. In addition to standard claim-level entailment, WiCE provides entailment judgments over sub-sentence units of the claim, and a minimal subset of evidence sentences that support each subclaim. To support this, we propose an automatic claim decomposition strategy using GPT-3.5 which we show is also effective at improving entailment models' performance on multiple datasets at test time. Finally, we show that real claims in our dataset involve challenging verification and retrieval problems that existing models fail to address.
Code (2)
Tasks
Fact CheckingNatural Language InferenceQuestion AnsweringRetrievalSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fool Me Twice: Entailment from Wikipedia Gamification
We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game. Gamification encourages adversarial examples, drastically lowering the number of examples…
RetrievalAutomated Fact-Checking of Claims from Wikipedia
Automated fact checking is becoming increasingly vital as both truthful and fallacious information accumulate online. Research on fact checking has benefited from large-scale datasets such as FEVER and SNLI. However, suc…
ArticlesFact CheckingFlashCheck: Exploration of Efficient Evidence Retrieval for Fast Fact-Checking
The advances in digital tools have led to the rampant spread of misinformation. While fact-checking aims to combat this, manual fact-checking is cumbersome and not scalable. It is essential for automated fact-checking to…
Fact CheckingFact VerificationMisinformationRetrievalQuanTemp: A real-world open-domain benchmark for fact-checking numerical claims
Automated fact checking has gained immense interest to tackle the growing misinformation in the digital era. Existing systems primarily focus on synthetic claims on Wikipedia, and noteworthy progress has also been made o…
Claim VerificationFact CheckingMisinformationAn End-to-End Multi-task Learning Model for Fact Checking
With huge amount of information generated every day on the web, fact checking is an important and challenging task which can help people identify the authenticity of most claims as well as providing evidences selected fr…
Common Sense ReasoningEntity LinkingFact CheckingFake News Detection+2