Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations
In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Information matrix via stochastic gradient descent, employing a weighting scheme that assigns heavier penalties for errors on frequent co-occurrences while still accounting for negative co-occurrence. Evaluation on word similarity and analogy tasks shows that LexVec matches and often outperforms state-of-the-art methods on many of these tasks.
Code (1)
Tasks
Word SimilaritySimilar Papers 제목 키워드 기반
Stochastic Subsampling for Factorizing Huge Matrices
We present a matrix-factorization algorithm that scales to input matrices with both huge number of rows and columns. Learned factors may be sparse or dense and/or non-negative, which makes our algorithm suitable for dict…
Dictionary LearningRandomized Algorithms for Symmetric Nonnegative Matrix Factorization
Symmetric Nonnegative Matrix Factorization (SymNMF) is a technique in data analysis and machine learning that approximates a symmetric matrix with a product of a nonnegative, low-rank matrix and its transpose. To design …
Graph ClusteringAn Empirical Comparison of Sampling Quality Metrics: A Case Study for Bayesian Nonnegative Matrix Factorization
In this work, we empirically explore the question: how can we assess the quality of samples from some target distribution? We assume that the samples are provided by some valid Monte Carlo procedure, so we are guaranteed…
DiversityvalidPartially Mutual Exclusive Softmax for Positive and Unlabeled data
In recent years, softmax together with its fast approximations has become the de-facto loss function for deep neural networks with multiclass predictions. However, softmax is used in many problems that do not fully fit t…
Language ModelingLanguage ModellingFree Energy Node Embedding via Generalized Skip-gram with Negative Sampling
A widely established set of unsupervised node embedding methods can be interpreted as consisting of two distinctive steps: i) the definition of a similarity matrix based on the graph of interest followed by ii) an explic…
ClusteringGPULink PredictionNode Classification+1