Figurative Language in Recognizing Textual Entailment
We introduce a collection of recognizing textual entailment (RTE) datasets focused on figurative language. We leverage five existing datasets annotated for a variety of figurative language -- simile, metaphor, and irony -- and frame them into over 12,500 RTE examples.We evaluate how well state-of-the-art models trained on popular RTE datasets capture different aspects of figurative language. Our results and analyses indicate that these models might not sufficiently capture figurative language, struggling to perform pragmatic inference and reasoning about world knowledge. Ultimately, our datasets provide a challenging testbed for evaluating RTE models.
Code (1)
Tasks
Natural Language InferenceRTEWorld KnowledgeSimilar Papers 제목 키워드 기반
FLUTE: Figurative Language Understanding through Textual Explanations
Figurative language understanding has been recently framed as a recognizing textual entailment (RTE) task (a.k.a. natural language inference, or NLI). However, similar to classical RTE/NLI datasets, the current benchmark…
Natural Language InferenceRTEUnderstanding Figurative Meaning through Explainable Visual Entailment
Large Vision-Language Models (VLMs) have demonstrated strong capabilities in tasks requiring a fine-grained understanding of literal meaning in images and text, such as visual question-answering or visual entailment. How…
Question AnsweringVisual EntailmentVisual Question AnsweringVisual Denotations for Recognizing Textual Entailment
In the logic approach to Recognizing Textual Entailment, identifying phrase-to-phrase semantic relations is still an unsolved problem. Resources such as the Paraphrase Database offer limited coverage despite their large …
Natural Language InferenceSemantic CompositionPerformance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
The quality of training data is one of the crucial problems when a learning-centered approach is employed. This paper proposes a new method to investigate the quality of a large corpus designed for the recognizing textua…
Natural Language InferenceRTESentence