Learning Word Representations with Hierarchical Sparse Coding
We propose a new method for learning word representations using hierarchical regularization in sparse coding inspired by the linguistic study of word meanings. We show an efficient learning algorithm based on stochastic proximal methods that is significantly faster than previous approaches, making it possible to perform hierarchical sparse coding on a corpus of billions of word tokens. Experiments on various benchmark tasks---word similarity ranking, analogies, sentence completion, and sentiment analysis---demonstrate that the method outperforms or is competitive with state-of-the-art methods. Our word representations are available at \url{http://www.ark.cs.cmu.edu/dyogatam/wordvecs/}.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence CompletionSentiment AnalysisWord SimilaritySimilar Papers 제목 키워드 기반
Inducing Sparse Coding and And-Or Grammar from Generator Network
We introduce an explainable generative model by applying sparse operation on the feature maps of the generator network. Meaningful hierarchical representations are obtained using the proposed generative model with sparse…
Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-o…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Part-Of-Speech Tagging+3Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-o…
Feature Engineeringnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+4Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference
Hierarchical predictive coding provides an interpretable framework for perception as error-driven inference in multi-layer generative models, while sparse coding imposes parsimonious latent representations through explic…
Neural Random Projections for Language Modelling
Neural network-based language models deal with data sparsity problems by mapping the large discrete space of words into a smaller continuous space of real-valued vectors. By learning distributed vector representations fo…
Language Modelling