Gradient Regularized Contrastive Learning for Continual Domain Adaptation
Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks to dynamic environments by machine learning algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. The obstacles in this problem are both domain shift and catastrophic forgetting. We propose Gradient Regularized Contrastive Learning (GRCL) to solve the obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforcing the gradient not to harm the discriminative ability of source features which can, in turn, benefit the adaptation ability of the model to target domains; (2) constraining the gradient not to increase the classification loss on old target domains, which enables the model to preserve the performance on old target domains when adapting to an in-coming target domain. Experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the other state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningDomain AdaptationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Gradient Regularized Contrastive Learning for Continual Domain Adaptation
Human beings can quickly adapt to environmental changes by leveraging learning experience. However, the poor ability of adapting to dynamic environments remains a major challenge for AI models. To better understand this …
Contrastive LearningDomain AdaptationRobust Mean Teacher for Continual and Gradual Test-Time Adaptation
Since experiencing domain shifts during test-time is inevitable in practice, test-time adaption (TTA) continues to adapt the model after deployment. Recently, the area of continual and gradual test-time adaptation (TTA) …
Contrastive LearningTest-time AdaptationPCoTTA: Continual Test-Time Adaptation for Multi-Task Point Cloud Understanding
In this paper, we present PCoTTA, an innovative, pioneering framework for Continual Test-Time Adaptation (CoTTA) in multi-task point cloud understanding, enhancing the model's transferability towards the continually chan…
Test-time AdaptationContinual Domain Adaptation on Aerial Images under Gradually Degrading Weather
Domain adaptation (DA) strives to mitigate the domain gap between the source domain where a model is trained, and the target domain where the model is deployed. When a deep learning model is deployed on an aerial platfor…
Domain AdaptationTest-time AdaptationDEJA VU: Continual Model Generalization For Unseen Domains
In real-world applications, deep learning models often run in non-stationary environments where the target data distribution continually shifts over time. There have been numerous domain adaptation (DA) methods in both o…
Data AugmentationDomain AdaptationDomain Generalizationmodel