On Romanization for Model Transfer Between Scripts in Neural Machine Translation
Transfer learning is a popular strategy to improve the quality of low-resource machine translation. For an optimal transfer of the embedding layer, the child and parent model should share a substantial part of the vocabulary. This is not the case when transferring to languages with a different script. We explore the benefit of romanization in this scenario. Our results show that romanization entails information loss and is thus not always superior to simpler vocabulary transfer methods, but can improve the transfer between related languages with different scripts. We compare two romanization tools and find that they exhibit different degrees of information loss, which affects translation quality. Finally, we extend romanization to the target side, showing that this can be a successful strategy when coupled with a simple deromanization model.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTransfer LearningTranslationSimilar Papers 제목 키워드 기반
One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models
Exposing latent lexical overlap, script romanization has emerged as an effective strategy for improving cross-lingual transfer (XLT) in multilingual language models (mLMs). Most prior work, however, focused on setups tha…
Cross-Lingual TransferRomanLens: Latent Romanization and its role in Multilinguality in LLMs
Large Language Models (LLMs) exhibit remarkable multilingual generalization despite being predominantly trained on English-centric corpora. A fundamental question arises: how do LLMs achieve such robust multilingual capa…
Language ModelingLanguage ModellingCUNI Systems for the WMT22 Czech-Ukrainian Translation Task
We present Charles University submissions to the WMT22 General Translation Shared Task on Czech-Ukrainian and Ukrainian-Czech machine translation. We present two constrained submissions based on block back-translation an…
Machine TranslationTranslationOne Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography
Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing systems. Prior work addresses this via script …
Cross-Lingual TransferCORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts
Naively assuming English as a source language may hinder cross-lingual transfer for many languages by failing to consider the importance of language contact. Some languages are more well-connected than others, and target…
Contrastive LearningCross-Lingual TransferZero-Shot Cross-Lingual Transfer