Word Re-Embedding via Manifold Dimensionality Retention
Word embeddings seek to recover a Euclidean metric space by mapping words into vectors, starting from words co-occurrences in a corpus. Word embeddings may underestimate the similarity between nearby words, and overestimate it between distant words in the Euclidean metric space. In this paper, we re-embed pre-trained word embeddings with a stage of manifold learning which retains dimensionality. We show that this approach is theoretically founded in the metric recovery paradigm, and empirically show that it can improve on state-of-the-art embeddings in word similarity tasks 0.5 - 5.0{\%} points depending on the original space.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNamed Entity Recognition (NER)Part-Of-Speech TaggingWord EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
Word, graph and manifold embedding from Markov processes
Continuous vector representations of words and objects appear to carry surprisingly rich semantic content. In this paper, we advance both the conceptual and theoretical understanding of word embeddings in three ways. Fir…
Dimensionality ReductionWord EmbeddingsGeometric Structures and Patterns of Meaning: A PHATE Manifold Analysis of Chinese Character Embeddings
We systematically investigate geometric patterns in Chinese character embeddings using PHATE manifold analysis. Through cross-validation across seven embedding models and eight dimensionality reduction methods, we observ…
Dimensionality ReductionDimensionality Reduction of Collective Motion by Principal Manifolds
While the existence of low-dimensional embedding manifolds has been shown in patterns of collective motion, the current battery of nonlinear dimensionality reduction methods are not amenable to the analysis of such manif…
Dimensionality ReductionMulti-Objective Genetic Programming for Manifold Learning: Balancing Quality and Dimensionality
Manifold learning techniques have become increasingly valuable as data continues to grow in size. By discovering a lower-dimensional representation (embedding) of the structure of a dataset, manifold learning algorithms …
Sparse Manifold Clustering and Embedding
We propose an algorithm called Sparse Manifold Clustering and Embedding (SMCE) for simultaneous clustering and dimensionality reduction of data lying in multiple nonlinear manifolds. Similar to most dimensionality reduct…
ClusteringDimensionality Reduction