Collaborative Learning for Enhanced Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) endeavors to bridge the gap between a model trained on a labeled source domain and its deployment in an unlabeled target domain. However, current high-performance models demand significant resources, making deployment costs prohibitive and highlighting the need for compact, yet effective models. For UDA of lightweight models, Knowledge Distillation (KD) leveraging a Teacher-Student framework could be a common approach, but we found that domain shift in UDA leads to a significant increase in non-salient parameters in the teacher model, degrading model's generalization ability and transferring misleading information to the student model. Interestingly, we observed that this phenomenon occurs considerably less in the student model. Driven by this insight, we introduce Collaborative Learning for UDA (CLDA), a method that updates the teacher's non-salient parameters using the student model and at the same time utilizes the updated teacher model to improve UDA performance of the student model. Experiments show consistent performance improvements for both student and teacher models. For example, in semantic segmentation, CLDA achieves an improvement of +0.7% mIoU for the teacher model and +1.4% mIoU for the student model compared to the baseline model in the GTA-to-Cityscapes datasets. In the Synthia-to-Cityscapes dataset, it achieves an improvement of +0.8% mIoU and +2.0% mIoU for the teacher and student models, respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationKnowledge DistillationSemantic SegmentationUnsupervised Domain AdaptationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Domain Adaptation with SAM-RefiSeR for Enhanced Brain Tumor Segmentation
Unsupervised Domain Adaptation with SAM-RefiSeR for Enhanced Brain Tumor Segmentation
Unsupervised Domain AdaptationBrain Tumor SegmentationMulti-Source Domain Adaptation with Collaborative Learning for Semantic Segmentation
Multi-source unsupervised domain adaptation~(MSDA) aims at adapting models trained on multiple labeled source domains to an unlabeled target domain. In this paper, we propose a novel multi-source domain adaptation framew…
Domain AdaptationMulti-Source Unsupervised Domain AdaptationSemantic SegmentationUnsupervised Domain AdaptationCollaborative and Adversarial Network for Unsupervised Domain Adaptation
In this paper, we propose a new unsupervised domain adaptation approach called Collaborative and Adversarial Network (CAN) through domain-collaborative and domain-adversarial training of neural networks. We use several d…
Domain AdaptationUnsupervised Domain AdaptationCollaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in practice: 1) only limited labeled samples are available due to expensive ann…
DiagnosticDomain AdaptationUnsupervised Domain AdaptationDUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception
Vehicle-to-Everything (V2X) collaborative perception is crucial for autonomous driving. However, achieving high-precision V2X perception requires a significant amount of annotated real-world data, which can always be exp…
Autonomous DrivingAutonomous VehiclesDomain Adaptation