An Analysis of Natural Language Inference Benchmarks through the Lens of Negation
Negation is underrepresented in existing natural language inference benchmarks. Additionally, one can often ignore the few negations in existing benchmarks and still make the right inference judgments. In this paper, we present a new benchmark for natural language inference in which negation plays a critical role. We also show that state-of-the-art transformers struggle making inference judgments with the new pairs.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceNegationSimilar Papers 제목 키워드 기반
Neural Semantic Encoders
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time a…
General ClassificationMachine TranslationNatural Language InferenceNatural Language Understanding+5Stacking Small Language Models for Generalizability
Recent advances show that large language models (LLMs) generalize strong performance across different natural language benchmarks. However, the large size of LLMs makes training and inference expensive and impractical to…
Stochastic Answer Networks for Natural Language Inference
We propose a stochastic answer network (SAN) to explore multi-step inference strategies in Natural Language Inference. Rather than directly predicting the results given the inputs, the model maintains a state and iterati…
Natural Language InferenceExploring Transitivity in Neural NLI Models through Veridicality
Despite the recent success of deep neural networks in natural language processing, the extent to which they can demonstrate human-like generalization capacities for natural language understanding remains unclear. We expl…
Natural Language InferenceNatural Language UnderstandingTesting the Generalization Power of Neural Network Models Across NLI Benchmarks
Neural network models have been very successful in natural language inference, with the best models reaching 90% accuracy in some benchmarks. However, the success of these models turns out to be largely benchmark specifi…
Natural Language InferenceTransfer Learning