Syntax-based Attention Model for Natural Language Inference
Introducing attentional mechanism in neural network is a powerful concept, and has achieved impressive results in many natural language processing tasks. However, most of the existing models impose attentional distribution on a flat topology, namely the entire input representation sequence. Clearly, any well-formed sentence has its accompanying syntactic tree structure, which is a much rich topology. Applying attention to such topology not only exploits the underlying syntax, but also makes attention more interpretable. In this paper, we explore this direction in the context of natural language inference. The results demonstrate its efficacy. We also perform extensive qualitative analysis, deriving insights and intuitions of why and how our model works.
Code (0)
등록된 구현이 없습니다.
Tasks
modelNatural Language InferenceSentenceSimilar Papers 제목 키워드 기반
Improving Natural Language Inference with a Pretrained Parser
We introduce a novel approach to incorporate syntax into natural language inference (NLI) models. Our method uses contextual token-level vector representations from a pretrained dependency parser. Like other contextual e…
Natural Language InferenceSG-Net: Syntax Guided Transformer for Language Representation
Understanding human language is one of the key themes of artificial intelligence. For language representation, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy texts and g…
Machine Reading ComprehensionMachine TranslationNatural Language InferenceReading Comprehension+1Enhancing Generalization in Natural Language Inference by Syntax
Pre-trained language models such as BERT have achieved the state-of-the-art performance on natural language inference (NLI). However, it has been shown that such models can be tricked by variations of surface patterns su…
Natural Language InferenceSentenceCooperative Learning of Disjoint Syntax and Semantics
There has been considerable attention devoted to models that learn to jointly infer an expression's syntactic structure and its semantics. Yet, \citet{NangiaB18} has recently shown that the current best systems fail to l…
Domain GeneralizationNatural Language InferenceSentiment AnalysisBeyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers
Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to syntactic structure. We introduce Syntax-informed Positional Embeddings (SiPE), which learns a lightweight syntactic …