IndT5: A Text-to-Text Transformer for 10 Indigenous Languages
Transformer language models have become fundamental components of natural language processing based pipelines. Although several Transformer models have been introduced to serve many languages, there is a shortage of models pre-trained for low-resource and Indigenous languages. In this work, we introduce IndT5, the first Transformer language model for Indigenous languages. To train IndT5, we build IndCorpus--a new dataset for ten Indigenous languages and Spanish. We also present the application of IndT5 to machine translation by investigating different approaches to translate between Spanish and the Indigenous languages as part of our contribution to the AmericasNLP 2021 Shared Task on Open Machine Translation. IndT5 and IndCorpus are publicly available for research
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
"It's how you do things that matters": Attending to Process to Better Serve Indigenous Communities with Language Technologies
Indigenous languages are historically under-served by Natural Language Processing (NLP) technologies, but this is changing for some languages with the recent scaling of large multilingual models and an increased focus by…
PositionImproving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learning
Machine translation (MT) involving Indigenous languages, including those possibly endangered, is challenging due to lack of sufficient parallel data. We describe an approach exploiting bilingual and multilingual pretrain…
Data AugmentationMachine TranslationTransfer LearningTranslationIndigenous language technologies in Canada: Assessment, challenges, and successes
In this article, we discuss which text, speech, and image technologies have been developed, and would be feasible to develop, for the approximately 60 Indigenous languages spoken in Canada. In particular, we concentrate …
Machine TranslationOptical Character RecognitionOptical Character Recognition (OCR)speaker-diarization+6Revitalization of Indigenous Languages through Pre-processing and Neural Machine Translation: The case of Inuktitut
Indigenous languages have been very challenging when dealing with NLP tasks and applications because of multiple reasons. These languages, in linguistic typology, are polysynthetic and highly inflected with rich morphoph…
Machine TranslationMorphological AnalysisTranslationCan Uniform Meaning Representation Help GPT-4 Translate from Indigenous Languages?
While ChatGPT and GPT-based models are able to effectively perform many tasks without additional fine-tuning, they struggle with related to extremely low-resource languages and indigenous languages. Uniform Meaning Repre…