Zero-shot Cross-lingual Transfer is Under-specified Optimization
Pretrained multilingual encoders enable zero-shot cross-lingual transfer performance, but often produce unreliable models that exhibit high performance variance on the target language. We postulate that high variance results from zero-shot cross-lingual transfer solving an under-specified optimization problem. We show that the source language monolingual model and source + target bilingual model are linearly connected using a model interpolation, suggesting that the model struggles to identify good solutions for both source and target languages using the source language alone.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferZero-Shot Cross-Lingual TransferSimilar Papers 제목 키워드 기반
Zero-shot Cross-lingual Transfer is Under-specified Optimization
Pretrained multilingual encoders enable zero-shot cross-lingual transfer, but often produce unreliable models that exhibit high performance variance on the target language. We postulate that this high variance results fr…
Cross-Lingual TransferZero-Shot Cross-Lingual TransferRealistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent MultiEURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual tr…
Cross-Lingual TransferTopic ClassificationTranslationZero-Shot Cross-Lingual TransferRealistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
We consider zero-shot cross-lingual transfer in legal topic classification using the recent Multi-EURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual t…
ClassificationCross-Lingual TransferTopic ClassificationTranslation+1Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer
Zero-shot cross-lingual transfer is a central task in multilingual NLP, allowing models trained in languages with more sufficient training resources to generalize to other low-resource languages. Earlier efforts on this …
Cross-Lingual TransferMultilingual NLPZero-Shot Cross-Lingual TransferZero-Shot Cross-Lingual Transfer is a Hard Baseline to Beat in German Fine-Grained Entity Typing
The training of NLP models often requires large amounts of labelled training data, which makes it difficult to expand existing models to new languages. While zero-shot cross-lingual transfer relies on multilingual word e…
Cross-Lingual TransferEntity TypingMultilingual Word Embeddingsnamed-entity-recognition+4