MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
NLP in the age of monolithic large language models is approaching its limits in terms of size and information that can be handled. The trend goes to modularization, a necessary step into the direction of designing smaller sub-networks and components with specialized functionality. In this paper, we present the MAMMOTH toolkit: a framework designed for training massively multilingual modular machine translation systems at scale, initially derived from OpenNMT-py and then adapted to ensure efficient training across computation clusters. We showcase its efficiency across clusters of A100 and V100 NVIDIA GPUs, and discuss our design philosophy and plans for future information. The toolkit is publicly available online.
Code (1)
Tasks
Machine TranslationPhilosophyTranslationSimilar Papers 제목 키워드 기반
Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data
This paper investigates a critical design decision in the practice of massively multilingual continual pre-training -- the inclusion of parallel data. Specifically, we study the impact of bilingual translation data for m…
TranslationMassively Multilingual Neural Machine Translation
Multilingual neural machine translation (NMT) enables training a single model that supports translation from multiple source languages into multiple target languages. In this paper, we push the limits of multilingual NMT…
Machine TranslationNMTTranslationCoVoST 2 and Massively Multilingual Speech-to-Text Translation
Speech translation has recently become an increasingly popular topic of research, partly due to the development of benchmark datasets. Nevertheless, current datasets cover a limited number of languages. With the aim to f…
Machine Translationspeech-recognitionSpeech RecognitionSpeech-to-Text+2Multilingual Neural Machine Translation
The advent of neural machine translation (NMT) has opened up exciting research in building multilingual translation systems i.e. translation models that can handle more than one language pair. Many advances have been mad…
Machine TranslationNMTTransfer LearningTranslationEvaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model. Its improved translation perf…
Cross-Lingual TransferMachine TranslationNMTTransfer Learning+1