WOVe: Incorporating Word Order in GloVe Word Embeddings
Word vector representations open up new opportunities to extract useful information from unstructured text. Defining a word as a vector made it easy for the machine learning algorithms to understand a text and extract information from. Word vector representations have been used in many applications such word synonyms, word analogy, syntactic parsing, and many others. GloVe, based on word contexts and matrix vectorization, is an ef-fective vector-learning algorithm. It improves on previous vector-learning algorithms. However, the GloVe model fails to explicitly consider the order in which words appear within their contexts. In this paper, multiple methods of incorporating word order in GloVe word embeddings are proposed. Experimental results show that our Word Order Vector (WOVe) word embeddings approach outperforms unmodified GloVe on the natural lan-guage tasks of analogy completion and word similarity. WOVe with direct concatenation slightly outperformed GloVe on the word similarity task, increasing average rank by 2%. However, it greatly improved on the GloVe baseline on a word analogy task, achieving an average 36.34% improvement in accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsWord SimilarityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Source-Criticism Debiasing Method for GloVe Embeddings
It is well-documented that word embeddings trained on large public corpora consistently exhibit known human social biases. Although many methods for debiasing exist, almost all fixate on completely eliminating biased inf…
Word EmbeddingsAn Automated Method to Enrich Consumer Health Vocabularies Using GloVe Word Embeddings and An Auxiliary Lexical Resource
Background: Clear language makes communication easier between any two parties. A layman may have difficulty communicating with a professional due to not understanding the specialized terms common to the domain. In health…
Word EmbeddingsSemGloVe: Semantic Co-occurrences for GloVe from BERT
GloVe learns word embeddings by leveraging statistical information from word co-occurrence matrices. However, word pairs in the matrices are extracted from a predefined local context window, which might lead to limited w…
Language ModelingLanguage ModellingWord EmbeddingsWord SimilarityExtremal GloVe: Theoretically Accurate Distributed Word Embedding by Tail Inference
Distributed word embeddings such as Word2Vec and GloVe have been widely adopted in industrial context settings. Major technical applications of GloVe include recommender systems and natural language processing. The funda…
Recommendation SystemsWord EmbeddingsExponential Family Word Embeddings: An Iterative Approach for Learning Word Vectors
GloVe and Skip-gram word embedding methods learn word vectors by decomposing a denoised matrix of word co-occurrences into a product of low-rank matrices. In this work, we propose an iterative algorithm for computing wor…
Word Embeddings