Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise
Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity. However, current approaches that operate in the embedding space do not take surface similarity into account. This work presents a simple yet effective strategy to imrove cross-lingual transfer between closely related varieties. We propose to augment the data of the high-resource source language with character-level noise to make the model more robust towards spelling variations. Our strategy shows consistent improvements over several languages and tasks: Zero-shot transfer of POS tagging and topic identification between language varieties from the Finnic, West and North Germanic, and Western Romance language branches. Our work provides evidence for the usefulness of simple surface-level noise in improving transfer between language varieties.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferPOSPOS TaggingZero-Shot Cross-Lingual TransferSimilar Papers 제목 키워드 기반
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer. However, these multilingual encoders do not precisely align words and phrases ac…
Cross-Lingual TransferSentencetext-classificationText Classification+3DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer
Zero-shot cross-lingual transfer is promising, however has been shown to be sub-optimal, with inferior transfer performance across low-resource languages. In this work, we envision languages as domains for improving zero…
Cross-Lingual TransferZero-Shot Cross-Lingual TransferZero-shot Cross-lingual Transfer is Under-specified Optimization
Pretrained multilingual encoders enable zero-shot cross-lingual transfer, but often produce unreliable models that exhibit high performance variance on the target language. We postulate that this high variance results fr…
Cross-Lingual TransferZero-Shot Cross-Lingual TransferA Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning
Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages. To disengage from these dependencies, researchers have…
Cross-Lingual Transfertext-classificationText ClassificationTransfer Learning+2Monolingual and Cross-lingual Zero-shot Style Transfer
We introduce the task of zero-shot style transfer between different languages. Our training data includes multilingual parallel corpora, but does not contain any parallel sentences between styles, similarly to the recent…
Machine TranslationStyle TransferTranslation