Deep Generative Model for Joint Alignment and Word Representation
This work exploits translation data as a source of semantically relevant learning signal for models of word representation. In particular, we exploit equivalence through translation as a form of distributed context and jointly learn how to embed and align with a deep generative model. Our EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. Besides, it embeds words as posterior probability densities, rather than point estimates, which allows us to compare words in context using a measure of overlap between distributions (e.g. KL divergence). We investigate our model's performance on a range of lexical semantics tasks achieving competitive results on several standard benchmarks including natural language inference, paraphrasing, and text similarity.
Code (1)
Tasks
Natural Language Inferencetext similarityTranslationSimilar Papers 제목 키워드 기반
Consistent Alignment of Word Embedding Models
Word embedding models offer continuous vector representations that can capture rich contextual semantics based on their word co-occurrence patterns. While these word vectors can provide very effective features used in ma…
ClusteringImproving Word Alignment of Rare Words with Word Embeddings
We address the problem of inducing word alignment for language pairs by developing an unsupervised model with the capability of getting applied to other generative alignment models. We approach the task by: i)proposing a…
Machine TranslationSentenceWord AlignmentWord EmbeddingsLarge-scale Word Alignment Using Soft Dependency Cohesion Constraints
Dependency cohesion refers to the observation that phrases dominated by disjoint dependency subtrees in the source language generally do not overlap in the target language. It has been verified to be a useful constraint …
Machine TranslationTranslationWord AlignmentNeural Text Classification by Jointly Learning to Cluster and Align
Distributional text clustering delivers semantically informative representations and captures the relevance between each word and semantic clustering centroids. We extend the neural text clustering approach to text class…
ClassificationClusteringGeneral Classificationtext-classification+3Generative latent neural models for automatic word alignment
Word alignments identify translational correspondences between words in a parallel sentence pair and are used, for instance, to learn bilingual dictionaries, to train statistical machine translation systems or to perform…
Machine TranslationSentenceText GenerationTranslation+1