Demystifying Learning of Unsupervised Neural Machine Translation
Unsupervised Neural Machine Translation or UNMT has received great attention in recent years. Though tremendous empirical improvements have been achieved, there still lacks theory-oriented investigation and thus some fundamental questions like \textit{why} certain training protocol can work or not under \textit{what} circumstances have not yet been well understood. This paper attempts to provide theoretical insights for the above questions. Specifically, following the methodology of comparative study, we leverage two perspectives, i) \textit{marginal likelihood maximization} and ii) \textit{mutual information} from information theory, to understand the different learning effects from the standard training protocol and its variants. Our detailed analyses reveal several critical conditions for the successful training of UNMT.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation
Recent work achieved remarkable results in training neural machine translation (NMT) systems in a fully unsupervised way, with new and dedicated architectures that rely on monolingual corpora only. In this work, we propo…
Machine TranslationNMTTranslationUnsupervised Machine TranslationUnsupervised Neural Machine Translation with Universal Grammar
Machine translation usually relies on parallel corpora to provide parallel signals for training. The advent of unsupervised machine translation has brought machine translation away from this reliance, though performance …
Machine TranslationTranslationUnsupervised Machine TranslationStudy on Unsupervised Statistical Machine Translation for Backtranslation
Machine Translation systems have drastically improved over the years for several language pairs. Monolingual data is often used to generate synthetic sentences to augment the training data which has shown to improve the …
Machine TranslationTranslationUnsupervised Machine TranslationThe LMU Munich Unsupervised Machine Translation System for WMT19
We describe LMU Munich{'}s machine translation system for German→Czech translation which was used to participate in the WMT19 shared task on unsupervised news translation. We train our model using monolingual data only…
DenoisingLanguage ModelingLanguage ModellingMachine Translation+3Supervised and Unsupervised Machine Translation for Myanmar-English and Khmer-English
This paper presents the NICT{'}s supervised and unsupervised machine translation systems for the WAT2019 Myanmar-English and Khmer-English translation tasks. For all the translation directions, we built state-of-the-art …
Machine TranslationNMTTranslationUnsupervised Machine Translation