Surprisingly Simple Adapter Ensembling for Zero-Shot Cross-Lingual Sequence Tagging
Adapters are parameter-efficient modules added to pretrained Transformer models that facilitate cross-lingual transfer. Language adapters and task adapters can be separately trained and zero-shot transfer is enabled by pairing the language adapter in the target language with a task adapter trained on a high-resource language. However, there are many languages and dialects for which training language adapters would be difficult. In this work, we present a simple and efficient ensembling technique to transfer task knowledge to unseen target languages for which no language adapters exist. We compute a uniformly-weighted ensemble model over the top language adapters based on how well they perform on the test set of a high-resource language. We outperform the state-of-the-art model for this specific setting on named entity recognition (NER) and part-of-speech tagging (POS), across nine typologically diverse languages with relative performance improvements of up to $29\%$ and $9\%$ on NER and POS, respectively, on select target languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERPart-Of-Speech TaggingPOSMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
What the Weight?! A Unified Framework for Zero-Shot Knowledge Composition
The knowledge encapsulated in a model is the core factor determining its final performance on downstream tasks. Much research in NLP has focused on efficient methods for storing and adapting different types of knowledge,…
BenchmarkingPointCLIP: Point Cloud Understanding by CLIP
Recently, zero-shot and few-shot learning via Contrastive Vision-Language Pre-training (CLIP) have shown inspirational performance on 2D visual recognition, which learns to match images with their corresponding texts in …
3D Open-Vocabulary Instance SegmentationFew-Shot LearningOpen Vocabulary Object DetectionTraining-free 3D Part Segmentation+5ZGUL: Zero-shot Generalization to Unseen Languages using Multi-source Ensembling of Language Adapters
We tackle the problem of zero-shot cross-lingual transfer in NLP tasks via the use of language adapters (LAs). Most of the earlier works have explored training with adapter of a single source (often English), and testing…
Cross-Lingual TransferLanguage ModellingNERPOS+3Snapshot Ensembles: Train 1, get M for free
Ensembles of neural networks are known to be much more robust and accurate than individual networks. However, training multiple deep networks for model averaging is computationally expensive. In this paper, we propose a …
Conceptrol: Concept Control of Zero-shot Personalized Image Generation
Personalized image generation with text-to-image diffusion models generates unseen images based on reference image content. Zero-shot adapter methods such as IP-Adapter and OminiControl are especially interesting because…
Image GenerationPersonalized Image Generation