Universal Neural Machine Translation for Extremely Low Resource Languages
In this paper, we propose a new universal machine translation approach focusing on languages with a limited amount of parallel data. Our proposed approach utilizes a transfer-learning approach to share lexical and sentence level representations across multiple source languages into one target language. The lexical part is shared through a Universal Lexical Representation to support multilingual word-level sharing. The sentence-level sharing is represented by a model of experts from all source languages that share the source encoders with all other languages. This enables the low-resource language to utilize the lexical and sentence representations of the higher resource languages. Our approach is able to achieve 23 BLEU on Romanian-English WMT2016 using a tiny parallel corpus of 6k sentences, compared to the 18 BLEU of strong baseline system which uses multilingual training and back-translation. Furthermore, we show that the proposed approach can achieve almost 20 BLEU on the same dataset through fine-tuning a pre-trained multi-lingual system in a zero-shot setting.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceTransfer LearningTranslationSimilar Papers 제목 키워드 기반
Using Interlinear Glosses as Pivot in Low-Resource Multilingual Machine Translation
We demonstrate a new approach to Neural Machine Translation (NMT) for low-resource languages using a ubiquitous linguistic resource, Interlinear Glossed Text (IGT). IGT represents a non-English sentence as a sequence of …
Machine TranslationNMTSentenceTranslationExtremely low-resource machine translation for closely related languages
An effective method to improve extremely low-resource neural machine translation is multilingual training, which can be improved by leveraging monolingual data to create synthetic bilingual corpora using the back-transla…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationTransfer Learning+1Learning-From-Mistakes Prompting for Indigenous Language Translation
Using large language models, this paper presents techniques to improve extremely low-resourced indigenous language translations. Our approaches are grounded in the use of (1) the presence of a datastore consisting of a l…
In-Context LearningTranslationMultilingual Neural Machine Translation with Language Clustering
Multilingual neural machine translation (NMT), which translates multiple languages using a single model, is of great practical importance due to its advantages in simplifying the training process, reducing online mainten…
ClusteringMachine TranslationNMTTranslationNeural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara
Low-resource languages present unique challenges to (neural) machine translation. We discuss the case of Bambara, a Mande language for which training data is scarce and requires significant amounts of pre-processing. Mor…
Machine TranslationTranslation