Out-of-Order Decoding for Robust Neural Machine Translation
Autoregressive recurrent neural decoders that generate sequences of tokens one-by-one and left-to-right are the workhorse of modern machine translation. In this work, we propose a new decoder architecture that can generate natural language sequences in an arbitrary order. Along with generating tokens from a given vocabulary, our model additionally learns to select the optimal position for each produced token. The proposed decoder architecture is fully compatible with the seq2seq framework and can be used as a drop-in replacement of any classical decoder. We demonstrate the performance of our new decoder on the IWSLT machine translation task as well as inspect and interpret the learned decoding patterns by analyzing how the model selects new positions for each subsequent token.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationPositionTranslationSimilar Papers 제목 키워드 기반
Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information
Non-autoregressive neural machine translation (NAT) generates each target word in parallel and has achieved promising inference acceleration. However, existing NAT models still have a big gap in translation quality compa…
Machine TranslationTranslationPhrase-based Machine Translation using Multiple Preordering Candidates
In this paper, we propose a new decoding method for phrase-based statistical machine translation which directly uses multiple preordering candidates as a graph structure. Compared with previous phrase-based decoding meth…
DecoderMachine TranslationTranslationSmart-Start Decoding for Neural Machine Translation
Most current neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to-left. In this work, we propose a novel method that breaks up the limitation of these decoding orders, ca…
Machine TranslationTranslationRule-based Reordering Space in Statistical Machine Translation
In Statistical Machine Translation (SMT), the constraints on word reorderings have a great impact on the set of potential translations that are explored. Notwithstanding computationnal issues, the reordering space of a S…
Machine TranslationTranslationLater-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
For extended periods of time, sequence generation models rely on beam search algorithm to generate output sequence. However, the correctness of beam search degrades when the a model is over-confident about a suboptimal p…
GPUMachine TranslationRerankingTranslation