Evaluating the Stability of Embedding-based Word Similarities
Word embeddings are increasingly being used as a tool to study word associations in specific corpora. However, it is unclear whether such embeddings reflect enduring properties of language or if they are sensitive to inconsequential variations in the source documents. We find that nearest-neighbor distances are highly sensitive to small changes in the training corpus for a variety of algorithms. For all methods, including specific documents in the training set can result in substantial variations. We show that these effects are more prominent for smaller training corpora. We recommend that users never rely on single embedding models for distance calculations, but rather average over multiple bootstrap samples, especially for small corpora.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic Textual SimilarityWord EmbeddingsSimilar Papers 제목 키워드 기반
Evaluation of Greek Word Embeddings
Since word embeddings have been the most popular input for many NLP tasks, evaluating their quality is of critical importance. Most research efforts are focusing on English word embeddings. This paper addresses the probl…
Word EmbeddingsEnhancing Translation Language Models with Word Embedding for Information Retrieval
In this paper, we explore the usage of Word Embedding semantic resources for Information Retrieval (IR) task. This embedding, produced by a shallow neural network, have been shown to catch semantic similarities between w…
Information RetrievalLanguage ModelingLanguage ModellingRetrieval+1Comparative Analysis of Word Embeddings for Capturing Word Similarities
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on dee…
Word EmbeddingsImproving Word Embeddings Using Kernel PCA
Word-based embedding approaches such as Word2Vec capture the meaning of words and relations between them, particularly well when trained with large text collections; however, they fail to do so with small datasets. Exten…
ArticlesSentenceSentence ClassificationWord Embeddings+1ABDN at SemEval-2018 Task 10: Recognising Discriminative Attributes using Context Embeddings and WordNet
This paper describes the system that we submitted for SemEval-2018 task 10: capturing discriminative attributes. Our system is built upon a simple idea of measuring the attribute word{'}s similarity with each of the two …
AttributeSemantic Textual SimilarityWord Embeddings