Morphological Priors for Probabilistic Neural Word Embeddings
Word embeddings allow natural language processing systems to share statistical information across related words. These embeddings are typically based on distributional statistics, making it difficult for them to generalize to rare or unseen words. We propose to improve word embeddings by incorporating morphological information, capturing shared sub-word features. Unlike previous work that constructs word embeddings directly from morphemes, we combine morphological and distributional information in a unified probabilistic framework, in which the word embedding is a latent variable. The morphological information provides a prior distribution on the latent word embeddings, which in turn condition a likelihood function over an observed corpus. This approach yields improvements on intrinsic word similarity evaluations, and also in the downstream task of part-of-speech tagging.
Code (0)
등록된 구현이 없습니다.
Tasks
Part-Of-Speech TaggingWord EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
Interpretable Word Embeddings via Informative Priors
Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital h…
Word EmbeddingsProbabilistic Embeddings with Laplacian Graph Priors
We introduce probabilistic embeddings using Laplacian priors (PELP). The proposed model enables incorporating graph side-information into static word embeddings. We theoretically show that the model unifies several previ…
Word EmbeddingsPBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding
We look into the task of \emph{generalizing} word embeddings: given a set of pre-trained word vectors over a finite vocabulary, the goal is to predict embedding vectors for out-of-vocabulary words, \emph{without} extra c…
POSPOS TaggingWord EmbeddingsWord SimilarityMorphological Word Embeddings
Linguistic similarity is multi-faceted. For instance, two words may be similar with respect to semantics, syntax, or morphology inter alia. Continuous word-embeddings have been shown to capture most of these shades of si…
Word EmbeddingsMorphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings
Neural machine translation has achieved impressive results in the last few years, but its success has been limited to settings with large amounts of parallel data. One way to improve NMT for lower-resource settings is to…
Low Resource NMTMachine TranslationNMTTranslation+2