Word Representations via Gaussian Embedding
Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages, including better capturing uncertainty about a representation and its relationships, expressing asymmetries more naturally than dot product or cosine similarity, and enabling more expressive parameterization of decision boundaries. This paper advocates for density-based distributed embeddings and presents a method for learning representations in the space of Gaussian distributions. We compare performance on various word embedding benchmarks, investigate the ability of these embeddings to model entailment and other asymmetric relationships, and explore novel properties of the representation.
Code (1)
Similar Papers 제목 키워드 기반
Multimodal Word Distributions
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich un…
Word EmbeddingsWord SimilarityLearning Multi-Sense Word Distributions using Approximate Kullback-Leibler Divergence
Learning word representations has garnered greater attention in the recent past due to its diverse text applications. Word embeddings encapsulate the syntactic and semantic regularities of sentences. Modelling word embed…
Word EmbeddingsWord SimilarityGaussian Hierarchical Latent Dirichlet Allocation: Bringing Polysemy Back
Topic models are widely used to discover the latent representation of a set of documents. The two canonical models are latent Dirichlet allocation, and Gaussian latent Dirichlet allocation, where the former uses multinom…
Topic ModelsMultivariate Gaussian Document Representation from Word Embeddings for Text Categorization
Recently, there has been a lot of activity in learning distributed representations of words in vector spaces. Although there are models capable of learning high-quality distributed representations of words, how to genera…
Text CategorizationWord EmbeddingsProbabilistic FastText for Multi-Sense Word Embeddings
We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information. In particular, we represent each word with a Gaussian mixture d…
Word EmbeddingsWord Similarity