Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages
We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods greatly outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. We show that our improvements are valid also in zero-shot/zero-resource scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationNMTTransfer LearningTranslationvalidSimilar Papers 제목 키워드 기반
Pivot Based Transfer Learning for Neural Machine Translation: CFILT IITB @ WMT 2021 Triangular MT
In this paper, we discuss the various techniques that we used to implement the Russian-Chinese machine translation system for the Triangular MT task at WMT 2021. Neural Machine translation systems based on transformer ar…
DecoderMachine TranslationTransfer LearningTranslationPersian-Spanish Low-Resource Statistical Machine Translation Through English as Pivot Language
This paper is an attempt to exclusively focus on investigating the pivot language technique in which a bridging language is utilized to increase the quality of the Persian-Spanish low-resource Statistical Machine Transla…
Machine TranslationSentenceTranslationNeural Machine Translation in Low-Resource Setting: a Case Study in English-Marathi Pair
In this paper and we explore different techniques of overcoming the challenges of low-resource in Neural Machine Translation (NMT) and specifically focusing on the case of English-Marathi NMT. NMT systems require a large…
Machine TranslationNMTTransfer LearningTranslationPivot Language for Low-Resource Machine Translation
Certain pairs of languages suffer from lack of a parallel corpus which is large in size and diverse in domain. One of the ways this is overcome is via use of a pivot language. In this paper we use Hindi as a pivot langua…
Machine TranslationTranslationTowards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer
Pivot-based neural machine translation (NMT) is commonly used in low-resource setups, especially for translation between non-English language pairs. It benefits from using high resource source-pivot and pivot-target lang…
Machine TranslationNMTreinforcement-learningReinforcement Learning (RL)+2