What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?
This document gives a knowledge-oriented analysis of about 20 interesting Recognizing Textual Entailment (RTE) examples, drawn from the 2005 RTE5 competition test set. The analysis ignores shallow statistical matching techniques between T and H, and rather asks: What would it take to reasonably infer that T implies H? What world knowledge would be needed for this task? Although such knowledge-intensive techniques have not had much success in RTE evaluations, ultimately an intelligent system should be expected to know and deploy this kind of world knowledge required to perform this kind of reasoning. The selected examples are typically ones which our RTE system (called BLUE) got wrong and ones which require world knowledge to answer. In particular, the analysis covers cases where there was near-perfect lexical overlap between T and H, yet the entailment was NO, i.e., examples that most likely all current RTE systems will have got wrong. A nice example is #341 (page 26), that requires inferring from "a river floods" that "a river overflows its banks". Seems it should be easy, right? Enjoy!
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceRTEWorld KnowledgeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on brittle formal logic. However, progress in…
Formal LogicKnowledge DistillationNatural Language InferencevalidCSReader at SemEval-2018 Task 11: Multiple Choice Question Answering as Textual Entailment
In this document we present an end-to-end machine reading comprehension system that solves multiple choice questions with a textual entailment perspective. Since some of the knowledge required is not explicitly mentioned…
Common Sense ReasoningLanguage ModellingMachine Reading ComprehensionMultiple-choice+4Textual Entailment with Dynamic Contrastive Learning for Zero-shot NER
In this paper, we study the problem of zero-shot NER, which aims at building a Named Entity Recognition (NER) system from scratch. It needs to identify the entities in the given sentences when we have zero token-level an…
Contrastive Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+3Explaining Answers with Entailment Trees
Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by showing the line of reasoning from what is known to the answer, rather than simply showing a fragment of textual evidence …
Language ModellingQuestion AnsweringvalidUniversal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start
A standard way to address different NLP problems is by first constructing a problem-specific dataset, then building a model to fit this dataset. To build the ultimate artificial intelligence, we desire a single machine t…
coreference-resolutionCoreference ResolutionNatural Language InferenceQuestion Answering