A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation
Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systematically compare the efficacy of several subword methods in promoting synergy and preventing interference across different linguistic typologies. Our findings show that subword regularisation boosts synergy in multilingual modelling, whereas BPE more effectively facilitates transfer during cross-lingual fine-tuning. Notably, our results suggest that differences in orthographic word boundary conventions (the morphological granularity of written words) may impede cross-lingual transfer more significantly than linguistic unrelatedness. Our study confirms that decisions around subword modelling can be key to optimising the benefits of multilingual modelling.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferMachine TranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models
Human understanding of language is robust to different word choices as far as they represent similar semantic concepts. To what extent does our human intuition transfer to language models, which represent all subwords as…
Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks
In zero-shot cross-lingual transfer, a supervised NLP task trained on a corpus in one language is directly applicable to another language without any additional training. A source of cross-lingual transfer can be as stra…
Cross-Lingual TransferLanguage ModelingLanguage ModellingMachine Reading Comprehension+7Subword Mapping and Anchoring across Languages
State-of-the-art multilingual systems rely on shared vocabularies that sufficiently cover all considered languages. To this end, a simple and frequently used approach makes use of subword vocabularies constructed jointly…
Cross-Lingual Natural Language InferenceMachine TranslationNatural Language InferenceTranslationInvestigating Lexical Sharing in Multilingual Machine Translation for Indian Languages
Multilingual language models have shown impressive cross-lingual transfer ability across a diverse set of languages and tasks. To improve the cross-lingual ability of these models, some strategies include transliteration…
Cross-Lingual TransferMachine TranslationTranslationTransliterationTransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data
Transliterating related languages that use different scripts into a common script is effective for improving crosslingual transfer in downstream tasks. However, this methodology often makes pretraining a model from scrat…
Transliteration