Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning
Recent studies have shown that code-switching data (CSD), in which multiple languages are mixed within the same context, can improve cross-lingual transfer and multilingual alignment in large language models (LLMs). However, existing studies primarily focus on bilingual transfer between English and a target language, leaving multilingual settings involving three or more languages largely unexplored. In this work, we investigate multilingual code-switching instruction tuning across four languages: English, Japanese, Korean, and Chinese. We evaluate multilingual understanding on Belebele. Our experiments show that simple sentence-level multilingual CSD consistently improves average multilingual performance across all four languages, indicating that multilingual code-switching can be effective beyond bilingual transfer settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferSimilar Papers 제목 키워드 기반
Bilingual Language Modeling, A transfer learning technique for Roman Urdu
Pretrained language models are now of widespread use in Natural Language Processing. Despite their success, applying them to Low Resource languages is still a huge challenge. Although Multilingual models hold great promi…
Cross-Lingual TransferLanguage ModelingLanguage ModellingMasked Language Modeling+1Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs
Automatic Speech Recognition (ASR) has become a key technology for human--AI interaction. However, code-switching ASR (CS-ASR) remains particularly challenging due to the severe scarcity of multilingual CS speech resourc…
Domain GeneralizationSpeech RecognitionCode-switching Language Modeling With Bilingual Word Embeddings: A Case Study for Egyptian Arabic-English
Code-switching (CS) is a widespread phenomenon among bilingual and multilingual societies. The lack of CS resources hinders the performance of many NLP tasks. In this work, we explore the potential use of bilingual word …
Language ModelingLanguage ModellingWord EmbeddingsStrategies of Code-switching in Human-Machine Dialogs
Most people are multilingual, and most multilinguals code-switch, yet the characteristics of code-switched language are not fully understood. We developed a chatbot capable of completing a Map Task with human participant…
Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer
Zero-shot cross-lingual transfer is a central task in multilingual NLP, allowing models trained in languages with more sufficient training resources to generalize to other low-resource languages. Earlier efforts on this …
Cross-Lingual TransferMultilingual NLPZero-Shot Cross-Lingual Transfer