Balancing Training for Multilingual Neural Machine Translation
When training multilingual machine translation (MT) models that can translate to/from multiple languages, we are faced with imbalanced training sets: some languages have much more training data than others. Standard practice is to up-sample less resourced languages to increase representation, and the degree of up-sampling has a large effect on the overall performance. In this paper, we propose a method that instead automatically learns how to weight training data through a data scorer that is optimized to maximize performance on all test languages. Experiments on two sets of languages under both one-to-many and many-to-one MT settings show our method not only consistently outperforms heuristic baselines in terms of average performance, but also offers flexible control over the performance of which languages are optimized.
Code (2)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Competence-based Curriculum Learning for Multilingual Machine Translation
Currently, multilingual machine translation is receiving more and more attention since it brings better performance for low resource languages (LRLs) and saves more space. However, existing multilingual machine translati…
Machine TranslationTranslationUncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training
Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adju…
Machine TranslationTranslationCauses and Cures for Interference in Multilingual Translation
Multilingual machine translation models can benefit from synergy between different language pairs, but also suffer from interference. While there is a growing number of sophisticated methods that aim to eliminate interfe…
Machine TranslationTranslationTowards Higher Pareto Frontier in Multilingual Machine Translation
Multilingual neural machine translation has witnessed remarkable progress in recent years. However, the long-tailed distribution of multilingual corpora poses a challenge of Pareto optimization, i.e., optimizing for some…
Knowledge DistillationMachine TranslationTranslationScaling Laws for Multilingual Neural Machine Translation
In this work, we provide a large-scale empirical study of the scaling properties of multilingual neural machine translation models. We examine how increases in the model size affect the model performance and investigate …
Machine TranslationTranslation