Self-Supervised Neural Machine Translation
We present a simple new method where an emergent NMT system is used for simultaneously selecting training data and learning internal NMT representations. This is done in a self-supervised way without parallel data, in such a way that both tasks enhance each other during training. The method is language independent, introduces no additional hyper-parameters, and achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training.
Code (1)
Tasks
Machine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
The LMU Munich Unsupervised Machine Translation System for WMT19
We describe LMU Munich{'}s machine translation system for German→Czech translation which was used to participate in the WMT19 shared task on unsupervised news translation. We train our model using monolingual data only…
DenoisingLanguage ModelingLanguage ModellingMachine Translation+3Self-supervised and Supervised Joint Training for Resource-rich Machine Translation
Self-supervised pre-training of text representations has been successfully applied to low-resource Neural Machine Translation (NMT). However, it usually fails to achieve notable gains on resource-rich NMT. In this paper,…
DecoderLow Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine Translation+2Self-Supervised Learning for Pairwise Data Refinement
Pairwise data automatically constructed from weakly supervised signals has been widely used for training deep learning models. Pairwise datasets such as parallel texts can have uneven quality levels overall, but usually …
Machine TranslationSelf-Supervised LearningTranslationSJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
In this paper, we introduced our joint team SJTU-NICT 's participation in the WMT 2020 machine translation shared task. In this shared task, we participated in four translation directions of three language pairs: English…
Collaborative FilteringLanguage ModelingLanguage ModellingMachine Translation+3SJTU-NICT’s Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
In this paper, we introduced our joint team SJTU-NICT ‘s participation in the WMT 2020 machine translation shared task. In this shared task, we participated in four translation directions of three language pairs: English…
Collaborative FilteringLanguage ModelingLanguage ModellingMachine Translation+3