Non-entailed subsequences as a challenge for natural language inference
Neural network models have shown great success at natural language inference (NLI), the task of determining whether a premise entails a hypothesis. However, recent studies suggest that these models may rely on fallible heuristics rather than deep language understanding. We introduce a challenge set to test whether NLI systems adopt one such heuristic: assuming that a sentence entails all of its subsequences, such as assuming that "Alice believes Mary is lying" entails "Alice believes Mary." We evaluate several competitive NLI models on this challenge set and find strong evidence that they do rely on the subsequence heuristic.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceSentenceSimilar Papers 제목 키워드 기반
Can current NLI systems handle German word order? Investigating language model performance on a new German challenge set of minimal pairs
Compared to English, German word order is freer and therefore poses additional challenges for natural language inference (NLI). We create WOGLI (Word Order in German Language Inference), the first adversarial NLI dataset…
Data AugmentationLanguage ModelingLanguage ModellingNatural Language InferenceAtomic-SNLI: Fine-Grained Natural Language Inference through Atomic Fact Decomposition
Current Natural Language Inference (NLI) systems primarily operate at the sentence level, providing black-box decisions that lack explanatory power. While atomic-level NLI offers a promising alternative by decomposing hy…
Natural Language InferenceIdentifying inherent disagreement in natural language inference
Natural language inference (NLI) is the task of determining whether a piece of text is entailed, contradicted by or unrelated to another piece of text. In this paper, we investigate how to tease systematic inferences (i.…
Natural Language InferenceExtracting and filtering paraphrases by bridging natural language inference and paraphrasing
Paraphrasing is a useful natural language processing task that can contribute to more diverse generated or translated texts. Natural language inference (NLI) and paraphrasing share some similarities and can benefit from …
Natural Language InferenceMCNS: Mining Causal Natural Structures Inside Time Series via A Novel Internal Causality Scheme
Causal inference permits us to discover covert relationships of various variables in time series. However, in most existing works, the variables mentioned above are the dimensions. The causality between dimensions could …
Causal InferenceTime SeriesTime Series Classification