A Survey on Recognizing Textual Entailment as an NLP Evaluation
Recognizing Textual Entailment (RTE) was proposed as a unified evaluation framework to compare semantic understanding of different NLP systems. In this survey paper, we provide an overview of different approaches for evaluating and understanding the reasoning capabilities of NLP systems. We then focus our discussion on RTE by highlighting prominent RTE datasets as well as advances in RTE dataset that focus on specific linguistic phenomena that can be used to evaluate NLP systems on a fine-grained level. We conclude by arguing that when evaluating NLP systems, the community should utilize newly introduced RTE datasets that focus on specific linguistic phenomena.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceRTESimilar Papers 제목 키워드 기반
Visual Denotations for Recognizing Textual Entailment
In the logic approach to Recognizing Textual Entailment, identifying phrase-to-phrase semantic relations is still an unsolved problem. Resources such as the Paraphrase Database offer limited coverage despite their large …
Natural Language InferenceSemantic CompositionEvaluating Compound Splitters Extrinsically with Textual Entailment
Traditionally, compound splitters are evaluated intrinsically on gold-standard data or extrinsically on the task of statistical machine translation. We explore a novel way for the extrinsic evaluation of compound splitte…
Information RetrievalMachine TranslationNatural Language InferenceSpeech Recognition+1蘊涵句型分析於改進中文文字蘊涵識別系統 (Entailment Analysis for Improving Chinese Recognizing Textual Entailment System) [In Chinese]
Evaluating Paraphrastic Robustness in Textual Entailment Models
We present PaRTE, a collection of 1,126 pairs of Recognizing Textual Entailment (RTE) examples to evaluate whether models are robust to paraphrasing. We posit that if RTE models understand language, their predictions sho…
Natural Language InferenceRTE