Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation
This paper highlights the impressive utility of multi-parallel corpora for transfer learning in a one-to-many low-resource neural machine translation (NMT) setting. We report on a systematic comparison of multistage fine-tuning configurations, consisting of (1) pre-training on an external large (209k{--}440k) parallel corpus for English and a helping target language, (2) mixed pre-training or fine-tuning on a mixture of the external and low-resource (18k) target parallel corpora, and (3) pure fine-tuning on the target parallel corpora. Our experiments confirm that multi-parallel corpora are extremely useful despite their scarcity and content-wise redundancy thus exhibiting the true power of multilingualism. Even when the helping target language is not one of the target languages of our concern, our multistage fine-tuning can give 3{--}9 BLEU score gains over a simple one-to-one model.
Code (0)
등록된 구현이 없습니다.
Tasks
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationNMTTransfer LearningTranslationSimilar Papers 제목 키워드 기반
Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation
This paper proposes a novel multilingual multistage fine-tuning approach for low-resource neural machine translation (NMT), taking a challenging Japanese--Russian pair for benchmarking. Although there are many solutions …
BenchmarkingDomain AdaptationLow Resource Neural Machine TranslationLow-Resource Neural Machine Translation+4NICT's participation to WAT 2019: Multilingualism and Multi-step Fine-Tuning for Low Resource NMT
In this paper we describe our submissions to WAT 2019 for the following tasks: English{--}Tamil translation and Russian{--}Japanese translation. Our team,{``}NICT-5{''}, focused on multilingual domain adaptation and back…
Domain AdaptationLow Resource NMTNMTTranslationMultistage Fine-tuning Strategies for Automatic Speech Recognition in Low-resource Languages
This paper presents a novel multistage fine-tuning strategy designed to enhance automatic speech recognition (ASR) performance in low-resource languages using OpenAI's Whisper model. In this approach we aim to build ASR …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionHow Vocabulary Sharing Facilitates Multilingualism in LLaMA?
Large Language Models (LLMs), often show strong performance on English tasks, while exhibiting limitations on other languages. What is an LLM's multilingual capability when it is trained only on certain languages? The un…
Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
Lectures translation is a case of spoken language translation and there is a lack of publicly available parallel corpora for this purpose. To address this, we examine a language independent framework for parallel corpus …
BenchmarkingDomain AdaptationMachine TranslationParallel Corpus Mining+2