Word Embedding based on Low-Rank Doubly Stochastic Matrix Decomposition
Word embedding, which encodes words into vectors, is an important starting point in natural language processing and commonly used in many text-based machine learning tasks. However, in most current word embedding approaches, the similarity in embedding space is not optimized in the learning. In this paper we propose a novel neighbor embedding method which directly learns an embedding simplex where the similarities between the mapped words are optimal in terms of minimal discrepancy to the input neighborhoods. Our method is built upon two-step random walks between words via topics and thus able to better reveal the topics among the words. Experiment results indicate that our method, compared with another existing word embedding approach, is more favorable for various queries.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Doubly Stochastic Neighbor Embedding on Spheres
Stochastic Neighbor Embedding (SNE) methods minimize the divergence between the similarity matrix of a high-dimensional data set and its counterpart from a low-dimensional embedding, leading to widely applied tools for d…
Data VisualizationLOTFormer: Doubly-Stochastic Linear Attention via Low-Rank Optimal Transport
Transformers have proven highly effective across modalities, but standard softmax attention scales quadratically with sequence length, limiting long context modeling. Linear attention mitigates this by approximating atte…
Doubly Stochastic Adaptive Neighbors Clustering via the Marcus Mapping
Clustering is a fundamental task in machine learning and data science, and similarity graph-based clustering is an important approach within this domain. Doubly stochastic symmetric similarity graphs provide numerous ben…
ClusteringComputational EfficiencyRow-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning
Decentralized learning often involves a weighted global loss with heterogeneous node weights $λ$. We revisit two natural strategies for incorporating these weights: (i) embedding them into the local losses to retain a un…
Geometry-aware Domain Adaptation for Unsupervised Alignment of Word Embeddings
We propose a novel manifold based geometric approach for learning unsupervised alignment of word embeddings between the source and the target languages. Our approach formulates the alignment learning problem as a domain …
Bilingual Lexicon InductionDomain AdaptationWord Embeddings