Enhancing Generalization in Natural Language Inference by Syntax
Pre-trained language models such as BERT have achieved the state-of-the-art performance on natural language inference (NLI). However, it has been shown that such models can be tricked by variations of surface patterns such as syntax. We investigate the use of dependency trees to enhance the generalization of BERT in the NLI task, leveraging on a graph convolutional network to represent a syntax-based matching graph with heterogeneous matching patterns. Experimental results show that, our syntax-based method largely enhance generalization of BERT on a test set where the sentence pair has high lexical overlap but diverse syntactic structures, and do not degrade performance on the standard test set. In other words, the proposed method makes BERT more robust on syntactic changes.
Code (1)
Tasks
Natural Language InferenceSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cooperative Learning of Disjoint Syntax and Semantics
There has been considerable attention devoted to models that learn to jointly infer an expression's syntactic structure and its semantics. Yet, \citet{NangiaB18} has recently shown that the current best systems fail to l…
Domain GeneralizationNatural Language InferenceSentiment AnalysisBroadening the Applicability of Conditional Syntax Splitting for Reasoning from Conditional Belief Bases
In nonmonotonic reasoning from conditional belief bases, an inference operator satisfying syntax splitting postulates allows for taking only the relevant parts of a belief base into account, provided that the belief base…
Multi Language Models for On-the-Fly Syntax Highlighting
Syntax highlighting is a critical feature in modern software development environments, enhancing code readability and developer productivity. However, delivering accurate highlighting in real time remains challenging for…
Few-Shot LearningFilling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs without Real Data Replay
Continual Semantic Parsing (CSP) aims to train parsers to convert natural language questions into SQL across tasks with limited annotated examples, adapting to the real-world scenario of dynamically updated databases. Pr…
Continual LearningLanguage ModelingLanguage ModellingLarge Language Model+1Syntax-based Attention Model for Natural Language Inference
Introducing attentional mechanism in neural network is a powerful concept, and has achieved impressive results in many natural language processing tasks. However, most of the existing models impose attentional distributi…
modelNatural Language InferenceSentence