BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer
Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages. In this work, we show that it is more effective to learn bilingual language pair adapters (BAs) when the goal is to optimize performance for a particular source-target transfer direction. Our novel BAD-X adapter framework trades off some modularity of dedicated LAs for improved transfer performance: we demonstrate consistent gains in three standard downstream tasks, and for the majority of evaluated low-resource languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferZero-Shot Cross-Lingual TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer
Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages. In this work, we …
Cross-Lingual TransferZero-Shot Cross-Lingual TransferMonolingual Adapters for Zero-Shot Neural Machine Translation
We propose a novel adapter layer formalism for adapting multilingual models. They are more parameter-efficient than existing adapter layers while obtaining as good or better performance. The layers are specific to one la…
Machine TranslationTranslationCross-Lingual Transfer with Target Language-Ready Task Adapters
Adapters have emerged as a modular and parameter-efficient approach to (zero-shot) cross-lingual transfer. The established MAD-X framework employs separate language and task adapters which can be arbitrarily combined to …
Cross-Lingual TransferZero-Shot Cross-Lingual TransferSurprisingly Simple Adapter Ensembling for Zero-Shot Cross-Lingual Sequence Tagging
Adapters are parameter-efficient modules added to pretrained Transformer models that facilitate cross-lingual transfer. Language adapters and task adapters can be separately trained and zero-shot transfer is enabled by p…
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+3Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer
Adapter modules have recently been used for efficient fine-tuning and language specialization of massively multilingual Transformers (MMTs), improving downstream zero-shot cross-lingual transfer. In this work, we propose…
Cross-Lingual TransferZero-Shot Cross-Lingual Transfer