On the Transferability of Massively Multilingual Pretrained Models in the Pretext of the Indo-Aryan and Tibeto-Burman Languages
In recent times, machine translation models can learn to perform implicit bridging between language pairs never seen explicitly during training and showing that transfer learning helps for languages with constrained resources. This work investigates the low resource machine translation via transfer learning from multilingual pre-trained models i.e. mBART-50 and mT5-base in the pretext of Indo-Aryan (Assamese and Bengali) and Tibeto-Burman (Manipuri) languages via finetuning as a downstream task. Assamese and Manipuri were absent in the pretraining of both mBART-50 and the mT5 models. However, the experimental results attest that the finetuning from these pre-trained models surpasses the multilingual model trained from scratch.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTransfer LearningTranslationSimilar Papers 제목 키워드 기반
ByT5 model for massively multilingual grapheme-to-phoneme conversion
In this study, we tackle massively multilingual grapheme-to-phoneme conversion through implementing G2P models based on ByT5. We have curated a G2P dataset from various sources that covers around 100 languages and traine…
Grapheme-to-Phoneme ConversionA General-Purpose Multilingual Document Encoder
Massively multilingual pretrained transformers (MMTs) have tremendously pushed the state of the art on multilingual NLP and cross-lingual transfer of NLP models in particular. While a large body of work leveraged MMTs to…
Cross-Lingual TransferDocument ClassificationLong-range modelingMultilingual NLP+2The Massively Multilingual Natural Language Understanding 2022 (MMNLU-22) Workshop and Competition
Despite recent progress in Natural Language Understanding (NLU), the creation of multilingual NLU systems remains a challenge. It is common to have NLU systems limited to a subset of languages due to lack of available da…
intent-classificationIntent ClassificationNatural Language Understandingslot-filling+1An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation
Massively multilingual neural machine translation (MMNMT) has been proven to enhance the translation quality of low-resource languages. In this paper, we empirically investigate the translation robustness of Indonesian-C…
Machine TranslationTranslationMassively Multilingual Lexical Specialization of Multilingual Transformers
While pretrained language models (PLMs) primarily serve as general-purpose text encoders that can be fine-tuned for a wide variety of downstream tasks, recent work has shown that they can also be rewired to produce high-…
Bilingual Lexicon InductionRetrievalSentenceSentence Retrieval+3