Improve Lexicon-based Word Embeddings By Word Sense Disambiguation
There have been some works that learn a lexicon together with the corpus to improve the word embeddings. However, they either model the lexicon separately but update the neural networks for both the corpus and the lexicon by the same likelihood, or minimize the distance between all of the synonym pairs in the lexicon. Such methods do not consider the relatedness and difference of the corpus and the lexicon, and may not be the best optimized. In this paper, we propose a novel method that considers the relatedness and difference of the corpus and the lexicon. It trains word embeddings by learning the corpus to predicate a word and its corresponding synonym under the context at the same time. For polysemous words, we use a word sense disambiguation filter to eliminate the synonyms that have different meanings for the context. To evaluate the proposed method, we compare the performance of the word embeddings trained by our proposed model, the control groups without the filter or the lexicon, and the prior works in the word similarity tasks and text classification task. The experimental results show that the proposed model provides better embeddings for polysemous words and improves the performance for text classification.
Code (0)
등록된 구현이 없습니다.
Tasks
General Classificationtext-classificationText ClassificationWord EmbeddingsWord Sense DisambiguationWord SimilaritySimilar Papers 제목 키워드 기반
Cross-Lingual Contextual Word Embeddings Mapping With Multi-Sense Words In Mind
Recent work in cross-lingual contextual word embedding learning cannot handle multi-sense words well. In this work, we explore the characteristics of contextual word embeddings and show the link between contextual word e…
Bilingual Lexicon InductionWord EmbeddingsTraining Word Sense Embeddings With Lexicon-based Regularization
We propose to improve word sense embeddings by enriching an automatic corpus-based method with lexicographic data. Information from a lexicon is introduced into the learning algorithm{'}s objective function through a reg…
Word EmbeddingsWord Sense DisambiguationAutomatically Linking Lexical Resources with Word Sense Embedding Models
Automatically learnt word sense embeddings are developed as an attempt to refine the capabilities of coarse word embeddings. The word sense representations obtained this way are, however, sensitive to underlying corpora …
Word EmbeddingsMultilingual Wordnet sense Ranking using nearest context
In this paper, we combine methods to estimate sense rankings from raw text with recent work on word embeddings to provide sense ranking estimates for the entries in the Open Multilingual WordNet (OMW). The existing Word2…
Word EmbeddingsContextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge
Understanding context-dependent variation in word meanings is a key aspect of human language comprehension supported by the lexicon. Lexicographic resources (e.g., WordNet) capture only some of this context-dependent var…
Word Embeddings