Learning Composition Models for Phrase Embeddings
Lexical embeddings can serve as useful representations for words for a variety of NLP tasks, but learning embeddings for phrases can be challenging. While separate embeddings are learned for each word, this is infeasible for every phrase. We construct phrase embeddings by learning how to compose word embeddings using features that capture phrase structure and context. We propose efficient unsupervised and task-specific learning objectives that scale our model to large datasets. We demonstrate improvements on both language modeling and several phrase semantic similarity tasks with various phrase lengths. We make the implementation of our model and the datasets available for general use.
Code (1)
Tasks
Language ModelingLanguage ModellingSemantic SimilaritySemantic Textual SimilarityWord EmbeddingsSimilar Papers 제목 키워드 기반
Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings
We present a novel method for jointly learning compositional and non-compositional phrase embeddings by adaptively weighting both types of embeddings using a compositionality scoring function. The scoring function is use…
Efficient, Compositional, Order-sensitive n-gram Embeddings
We propose ECO: a new way to generate embeddings for phrases that is Efficient, Compositional, and Order-sensitive. Our method creates decompositional embeddings for words offline and combines them to create new embeddin…
Word EmbeddingsWhen a Red Herring in Not a Red Herring: Using Compositional Methods to Detect Non-Compositional Phrases
Non-compositional phrases such as \textit{red herring} and weakly compositional phrases such as \textit{spelling bee} are an integral part of natural language (Sag, 2002). They are also the phrases that are difficult, or…
Phrase-based Unsupervised Machine Translation with Compositional Phrase Embeddings
This paper describes the University of Tartu{'}s submission to the unsupervised machine translation track of WMT18 news translation shared task. We build several baseline translation systems for both directions of the En…
Machine TranslationTranslationUnsupervised Machine TranslationLearning Compositionality Functions on Word Embeddings for Modelling Attribute Meaning in Adjective-Noun Phrases
Word embeddings have been shown to be highly effective in a variety of lexical semantic tasks. They tend to capture meaningful relational similarities between individual words, at the expense of lacking the capabilty of …
AttributeRelationWord Embeddings