Knowledge Enhanced Attention for Robust Natural Language Inference
Neural network models have been very successful at achieving high accuracy on natural language inference (NLI) tasks. However, as demonstrated in recent literature, when tested on some simple adversarial examples, most of the models suffer a significant drop in performance. This raises the concern about the robustness of NLI models. In this paper, we propose to make NLI models robust by incorporating external knowledge to the attention mechanism using a simple transformation. We apply the new attention to two popular types of NLI models: one is Transformer encoder, and the other is a decomposable model, and show that our method can significantly improve their robustness. Moreover, when combined with BERT pretraining, our method achieves the human-level performance on the adversarial SNLI data set.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration). Inspired by the masking strategy of BERT, ERNIE is designed to learn language r…
Chinese Named Entity RecognitionChinese Sentence Pair ClassificationChinese Sentiment AnalysisCloze Test+7Knowledge-Enhanced Natural Language Inference Based on Knowledge Graphs
Natural Language Inference (NLI) is a vital task in natural language processing. It aims to identify the logical relationship between two sentences. Most of the existing approaches make such inference based on semantic k…
Knowledge GraphsNatural Language InferenceRelationNeural Natural Language Inference Models Enhanced with External Knowledge
Modeling natural language inference is a very challenging task. With the availability of large annotated data, it has recently become feasible to train complex models such as neural-network-based inference models, which …
Natural Language InferenceExBERT: An External Knowledge Enhanced BERT for Natural Language Inference
Neural language representation models such as BERT, pre-trained on large-scale unstructured corpora lack explicit grounding to real-world commonsense knowledge and are often unable to remember facts required for reasonin…
Knowledge GraphsNatural Language InferenceSeveral Experiments on Investigating Pretraining and Knowledge-Enhanced Models for Natural Language Inference
Natural language inference (NLI) is among the most challenging tasks in natural language understanding. Recent work on unsupervised pretraining that leverages unsupervised signals such as language-model and sentence pred…
Language ModelingLanguage ModellingNatural Language InferenceNatural Language Understanding+1