RPD: A Distance Function Between Word Embeddings
It is well-understood that different algorithms, training processes, and corpora produce different word embeddings. However, less is known about the relation between different embedding spaces, i.e. how far different sets of embeddings deviate from each other. In this paper, we propose a novel metric called Relative pairwise inner Product Distance (RPD) to quantify the distance between different sets of word embeddings. This metric has a unified scale for comparing different sets of word embeddings. Based on the properties of RPD, we study the relations of word embeddings of different algorithms systematically and investigate the influence of different training processes and corpora. The results shed light on the poorly understood word embeddings and justify RPD as a measure of the distance of embedding spaces.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Distilled Wasserstein Learning for Word Embedding and Topic Modeling
We propose a novel Wasserstein method with a distillation mechanism, yielding joint learning of word embeddings and topics. The proposed method is based on the fact that the Euclidean distance between word embeddings may…
Mortality PredictionWord EmbeddingsSpeeding up Word Mover's Distance and its variants via properties of distances between embeddings
The Word Mover's Distance (WMD) proposed by Kusner et al. is a distance between documents that takes advantage of semantic relations among words that are captured by their embeddings. This distance proved to be quite eff…
Document ClassificationGeneral ClassificationDistance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddings
A measure of similarity between text embeddings can be considered adequate only if it adheres to the human perception of similarity between texts. In this paper, we introduce the distance-to-distance ratio (DDR), a novel…
Angular-Based Word Meta-Embedding Learning
Ensembling word embeddings to improve distributed word representations has shown good success for natural language processing tasks in recent years. These approaches either carry out straightforward mathematical operatio…
Meta-LearningWord EmbeddingsWord SimilarityA Theoretical Framework for Acoustic Neighbor Embeddings
This paper provides a theoretical framework for interpreting acoustic neighbor embeddings, which are representations of the phonetic content of variable-width audio or text in a fixed-dimensional embedding space. A proba…
Clustering