Generalizing Back-Translation in Neural Machine Translation
Back-translation - data augmentation by translating target monolingual data - is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German - English news translation task.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationMachine TranslationNMTSynthetic Data GenerationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Explaining and Generalizing Back-Translation through Wake-Sleep
Back-translation has become a commonly employed heuristic for semi-supervised neural machine translation. The technique is both straightforward to apply and has led to state-of-the-art results. In this work, we offer a p…
Machine TranslationTranslationDomain-independent Punctuation and Segmentation Insertion
Punctuation and segmentation is crucial in spoken language translation, as it has a strong impact to translation performance. However, the impact of rare or unknown words in the performance of punctuation and segmentatio…
Machine TranslationPOSSegmentationTranslationQuick Back-Translation for Unsupervised Machine Translation
The field of unsupervised machine translation has seen significant advancement from the marriage of the Transformer and the back-translation algorithm. The Transformer is a powerful generative model, and back-translation…
DecoderMachine TranslationTranslationUnsupervised Machine TranslationIterative Batch Back-Translation for Neural Machine Translation: A Conceptual Model
An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of back-translations of the target-side monolingual data. Recently, iterative…
Machine TranslationNMTTranslationStudy on Unsupervised Statistical Machine Translation for Backtranslation
Machine Translation systems have drastically improved over the years for several language pairs. Monolingual data is often used to generate synthetic sentences to augment the training data which has shown to improve the …
Machine TranslationTranslationUnsupervised Machine Translation