Synonym Detection Using Syntactic Dependency And Neural Embeddings
Recent advances on the Vector Space Model have significantly improved some NLP applications such as neural machine translation and natural language generation. Although word co-occurrences in context have been widely used in counting-/predicting-based distributional models, the role of syntactic dependencies in deriving distributional semantics has not yet been thoroughly investigated. By comparing various Vector Space Models in detecting synonyms in TOEFL, we systematically study the salience of syntactic dependencies in accounting for distributional similarity. We separate syntactic dependencies into different groups according to their various grammatical roles and then use context-counting to construct their corresponding raw and SVD-compressed matrices. Moreover, using the same training hyperparameters and corpora, we study typical neural embeddings in the evaluation. We further study the effectiveness of injecting human-compiled semantic knowledge into neural embeddings on computing distributional similarity. Our results show that the syntactically conditioned contexts can interpret lexical semantics better than the unconditioned ones, whereas retrofitting neural embeddings with semantic knowledge can significantly improve synonym detection.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationText GenerationSimilar Papers 제목 키워드 기반
Multilingual Irony Detection with Dependency Syntax and Neural Models
This paper presents an in-depth investigation of the effectiveness of dependency-based syntactic features on the irony detection task in a multilingual perspective (English, Spanish, French and Italian). It focuses on th…
Word EmbeddingsGiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks. However, BERT is only trained to predict missing words - either behind masks or in the next sen…
Semantic SimilaritySemantic Textual SimilaritySentenceUnsupervised Pre-training+1Multi-grained Label Refinement Network with Dependency Structures for Joint Intent Detection and Slot Filling
Slot filling and intent detection are two fundamental tasks in the field of natural language understanding. Due to the strong correlation between these two tasks, previous studies make efforts on modeling them with multi…
Graph AttentionIntent DetectionMulti-Task LearningNatural Language Understanding+3RGAT: A Deeper Look into Syntactic Dependency Information for Coreference Resolution
Although syntactic information is beneficial for many NLP tasks, combining it with contextual information between words to solve the coreference resolution problem needs to be further explored. In this paper, we propose …
coreference-resolutionCoreference ResolutionGraph AttentionGiBERT: Enhancing BERT with Linguistic Information using a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks. However, BERT is only trained to predict missing words – either through masking or next sentenc…
Semantic SimilaritySemantic Textual SimilaritySentenceUnsupervised Pre-training+1