The Obscure Limitation of Modular Multilingual Language Models
We expose the limitation of modular multilingual language models (MLMs) in multilingual inference scenarios with unknown languages. Existing evaluations of modular MLMs exclude the involvement of language identification (LID) modules, which obscures the performance of real-case multilingual scenarios of modular MLMs. In this work, we showcase the effect of adding LID on the multilingual evaluation of modular MLMs and provide discussions for closing the performance gap of caused by the pipelined approach of LID and modular MLMs.
Code (0)
등록된 구현이 없습니다.
Tasks
Language IdentificationSimilar Papers 제목 키워드 기반
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations
Multilingual sequence-to-sequence models perform poorly with increased language coverage and fail to consistently generate text in the correct target language in few-shot settings. To address these challenges, we propose…
HallucinationNatural Language UnderstandingModular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Multilingual sentence encoders are commonly obtained by training multilingual language models to map sentences from different languages into a shared semantic space. As such, they are subject to curse of multilinguality,…
Contrastive LearningMultiple-choiceSentenceSentence Embeddings+1UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover…
Machine TranslationIs Modularity Transferable? A Case Study through the Lens of Knowledge Distillation
The rise of Modular Deep Learning showcases its potential in various Natural Language Processing applications. Parameter-efficient fine-tuning (PEFT) modularity has been shown to work for various use cases, from domain a…
Domain AdaptationKnowledge DistillationLanguage ModelingLanguage Modelling+6Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks
Recent work has proposed explicitly inducing language-wise modularity in multilingual LMs via sparse fine-tuning (SFT) on per-language subnetworks as a means of better guiding cross-lingual sharing. In this work, we inve…