Boosting Unsupervised Machine Translation with Pseudo-Parallel Data
Even with the latest developments in deep learning and large-scale language modeling, the task of machine translation (MT) of low-resource languages remains a challenge. Neural MT systems can be trained in an unsupervised way without any translation resources but the quality lags behind, especially in truly low-resource conditions. We propose a training strategy that relies on pseudo-parallel sentence pairs mined from monolingual corpora in addition to synthetic sentence pairs back-translated from monolingual corpora. We experiment with different training schedules and reach an improvement of up to 14.5 BLEU points (English to Ukrainian) over a baseline trained on back-translated data only.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine TranslationSentenceTranslationUnsupervised Machine TranslationSimilar Papers 제목 키워드 기반
Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings
We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search …
Machine TranslationSentenceSentence EmbeddingsTranslationFiltering Back-Translated Data in Unsupervised Neural Machine Translation
Unsupervised neural machine translation (NMT) utilizes only monolingual data for training. The quality of back-translated data plays an important role in the performance of NMT systems. In back-translation, all generated…
Domain AdaptationMachine TranslationNMTSentence+1USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation
The vast majority of evaluation metrics for machine translation are supervised, i.e., (i) are trained on human scores, (ii) assume the existence of reference translations, or (iii) leverage parallel data. This hinders th…
Machine TranslationParallel Corpus MiningSentenceSentence Embeddings+1Contrastive Clustering to Mine Pseudo Parallel Data for Unsupervised Translation
Modern unsupervised machine translation systems mostly train their models by generating synthetic parallel training data from large unlabeled monolingual corpora of different languages through various means, such as iter…
ClusteringMachine TranslationTranslationUnsupervised Machine TranslationData Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings
Unsupervised cross-lingual word embedding (CLWE) methods learn a linear transformation matrix that maps two monolingual embedding spaces that are separately trained with monolingual corpora. This method relies on the ass…
Cross-Lingual Word EmbeddingsData AugmentationMachine TranslationTranslation+2