Improving Cross-lingual Text Classification with Zero-shot Instance-Weighting
Cross-lingual text classification (CLTC) is a challenging task made even harder still due to the lack of labeled data in low-resource languages. In this paper, we propose zero-shot instance-weighting, a general model-agnostic zero-shot learning framework for improving CLTC by leveraging source instance weighting. It adds a module on top of pre-trained language models for similarity computation of instance weights, thus aligning each source instance to the target language. During training, the framework utilizes gradient descent that is weighted by instance weights to update parameters. We evaluate this framework over seven target languages on three fundamental tasks and show its effectiveness and extensibility, by improving on F1 score up to 4% in single-source transfer and 8% in multi-source transfer. To the best of our knowledge, our method is the first to apply instance weighting in zero-shot CLTC. It is simple yet effective and easily extensible into multi-source transfer.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationtext-classificationText ClassificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Cross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training
Multilingual pre-trained language models (MPLMs) not only can handle tasks in different languages but also exhibit surprising zero-shot cross-lingual transferability. However, MPLMs usually are not able to achieve compar…
Language ModelingLanguage Modellingtext-classificationText ClassificationBeyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers
We explore cross-lingual transfer of register classification for web documents. Registers, that is, text varieties such as blogs or news are one of the primary predictors of linguistic variation and thus affect the autom…
ClassificationCross-Lingual TransferGeneral ClassificationZero-Shot Cross-Lingual TransferRealistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent MultiEURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual tr…
Cross-Lingual TransferTopic ClassificationTranslationZero-Shot Cross-Lingual TransferRealistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent Multi-EURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual t…
ClassificationCross-Lingual TransferTopic ClassificationTranslation+1Zero-Shot Cross-Lingual Transfer in Legal Domain Using Transformer Models
Zero-shot cross-lingual transfer is an important feature in modern NLP models and architectures to support low-resource languages. In this work, We study zero-shot cross-lingual transfer from English to French and German…
ClassificationCross-Lingual TransferLanguage ModelingLanguage Modelling+6