An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition
Cross-lingual named entity recognition task is one of the critical problem for evaluating the potential transfer learning techniques on low resource languages. Knowledge distillation using pre-trained multilingual language models between source and target languages have shown their superiority. However, existing cross-lingual distillation models merely consider the potential transferability between two identical single tasks across both domain. Other possible auxiliary tasks to improve the learning performance have not been fully investigated. In this study, based on the knowledge distillation framework and multi-task learning, we introduce the similarity metric model as an auxiliary task to improve the cross-lingual NER performance on target domain. Specifically, an entity recognizer and a similarity evaluator teachers are first trained in parallel from the source domain. Then, two tasks in the student model are supervised by the two teachers simultaneously. Empirical studies on the datasets across 7 different languages confirm the effectiveness of the proposed model.
Code (1)
Tasks
Cross-Lingual NERKnowledge DistillationMulti-Task Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation
Unsupervised domain adaptation (UDA) seeks to alleviate the problem of domain shift between the distribution of unlabeled data from the target domain w.r.t. labeled data from the source domain. While the single-target UD…
Domain AdaptationKnowledge DistillationMulti-target Domain AdaptationSpecificity+1An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition
Cross-lingual named entity recognition task is one of the critical problems for evaluating the potential transfer learning techniques on low resource languages. Knowledge distillation using pre-trained multilingual langu…
Cross-Lingual NERKnowledge DistillationMulti-Task Learningnamed-entity-recognition+4CLDA-YOLO: Visual Contrastive Learning Based Domain Adaptive YOLO Detector
Unsupervised domain adaptive (UDA) algorithms can markedly enhance the performance of object detectors under conditions of domain shifts, thereby reducing the necessity for extensive labeling and retraining. Current doma…
Contrastive LearningData Augmentationobject-detectionObject DetectionContrastive Knowledge Amalgamation for Unsupervised Image Classification
Knowledge amalgamation (KA) aims to learn a compact student model to handle the joint objective from multiple teacher models that are are specialized for their own tasks respectively. Current methods focus on coarsely al…
Classificationimage-classificationImage ClassificationUnsupervised Image ClassificationUnsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training
In this work, we tackle the problem of unsupervised domain adaptation (UDA) for video action recognition. Our approach, which we call UNITE, uses an image teacher model to adapt a video student model to the target domain…
Action RecognitionDomain AdaptationTemporal Action LocalizationUnsupervised Domain Adaptation