Auto-Encoding Variational Neural Machine Translation
We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation and is parameterised by neural networks. We perform efficient training using amortised variational inference and reparameterised gradients. Additionally, we discuss the statistical implications of joint modelling and propose an efficient approximation to maximum a posteriori decoding for fast test-time predictions. We demonstrate the effectiveness of our model in three machine translation scenarios: in-domain training, mixed-domain training, and learning from a mix of gold-standard and synthetic data. Our experiments show consistently that our joint formulation outperforms conditional modelling (i.e. standard neural machine translation) in all such scenarios.
Code (1)
Tasks
Machine TranslationSentenceTranslationVariational InferenceSimilar Papers 제목 키워드 기반
(Self-Attentive) Autoencoder-based Universal Language Representation for Machine Translation
Universal language representation is the holy grail in machine translation (MT). Thanks to the new neural MT approach, it seems that there are good perspectives towards this goal. In this paper, we propose a new architec…
Machine TranslationSentenceTranslationUsing Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation
Non-autoregressive neural machine translation (NAT) usually employs sequence-level knowledge distillation using autoregressive neural machine translation (AT) as its teacher model. However, a NAT model often outputs shor…
Knowledge DistillationMachine TranslationTranslationVariational Recurrent Neural Machine Translation
Partially inspired by successful applications of variational recurrent neural networks, we propose a novel variational recurrent neural machine translation (VRNMT) model in this paper. Different from the variational NMT,…
DecoderMachine TranslationNMTSentence+1AVT: Unsupervised Learning of Transformation Equivariant Representations by Autoencoding Variational Transformations
The learning of Transformation-Equivariant Representations (TERs), which is introduced by Hinton et al. \cite{hinton2011transforming}, has been considered as a principle to reveal visual structures under various transfor…
DecoderVariational Neural Machine Translation
Models of neural machine translation are often from a discriminative family of encoderdecoders that learn a conditional distribution of a target sentence given a source sentence. In this paper, we propose a variational m…
DecoderMachine TranslationSentenceTranslation