Modeling Latent Sentence Structure in Neural Machine Translation
Recently it was shown that linguistic structure predicted by a supervised parser can be beneficial for neural machine translation (NMT). In this work we investigate a more challenging setup: we incorporate sentence structure as a latent variable in a standard NMT encoder-decoder and induce it in such a way as to benefit the translation task. We consider German-English and Japanese-English translation benchmarks and observe that when using RNN encoders the model makes no or very limited use of the structure induction apparatus. In contrast, CNN and word-embedding-based encoders rely on latent graphs and force them to encode useful, potentially long-distance, dependencies.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationNMTSentenceTranslationSimilar Papers 제목 키워드 기반
Inducing Constituency Trees through Neural Machine Translation
Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task. Recent advances in latent tree learning have made it possible to recover moderately high quality tree str…
Language ModelingLanguage ModellingMachine TranslationTranslationShared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation
Non-autoregressive neural machine translation (NAT) offers substantial translation speed up compared to autoregressive neural machine translation (AT) at the cost of translation quality. Latent variable modeling has emer…
Machine TranslationTranslationVariational Neural Machine Translation with Normalizing Flows
Variational Neural Machine Translation (VNMT) is an attractive framework for modeling the generation of target translations, conditioned not only on the source sentence but also on some latent random variables. The laten…
Machine TranslationNMTSentenceTranslationSequence-to-Dependency Neural Machine Translation
Nowadays a typical Neural Machine Translation (NMT) model generates translations from left to right as a linear sequence, during which latent syntactic structures of the target sentences are not explicitly concerned. Ins…
Machine TranslationNMTTranslationNeural Machine Translation with Latent Semantic of Image and Text
Although attention-based Neural Machine Translation have achieved great success, attention-mechanism cannot capture the entire meaning of the source sentence because the attention mechanism generates a target word depend…
Machine TranslationSentenceTranslation