MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation
For unsupervised domain adaptation (UDA), to alleviate the effect of domain shift, many approaches align the source and target domains in the feature space by adversarial learning or by explicitly aligning their statistics. However, the optimization objective of such domain alignment is generally not coordinated with that of the object classification task itself such that their descent directions for optimization may be inconsistent. This will reduce the effectiveness of domain alignment in improving the performance of UDA. In this paper, we aim to study and alleviate the optimization inconsistency problem between the domain alignment and classification tasks. We address this by proposing an effective meta-optimization based strategy dubbed MetaAlign, where we treat the domain alignment objective and the classification objective as the meta-train and meta-test tasks in a meta-learning scheme. MetaAlign encourages both tasks to be optimized in a coordinated way, which maximizes the inner product of the gradients of the two tasks during training. Experimental results demonstrate the effectiveness of our proposed method on top of various alignment-based baseline approaches, for tasks of object classification and object detection. MetaAlign helps achieve the state-of-the-art performance.
Code (1)
Tasks
ClassificationDomain AdaptationGeneral ClassificationMeta-LearningObjectobject-detectionObject DetectionUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time
Large Language Models (LLMs) acquire extensive knowledge and remarkable abilities from extensive text corpora, making them powerful tools for various applications. To make LLMs more usable, aligning them with human prefe…
DiversityMetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models
Recent advancements in large language models (LLMs) focus on aligning to heterogeneous human expectations and values via multi-objective preference alignment. However, existing methods are dependent on the policy model p…
GPUIn-Context LearningMinimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation
In this work, we face the problem of unsupervised domain adaptation with a novel deep learning approach which leverages on our finding that entropy minimization is induced by the optimal alignment of second order statist…
Domain AdaptationGeneral ClassificationUnsupervised Domain AdaptationDomain Adaptation and Image Classification via Deep Conditional Adaptation Network
Unsupervised domain adaptation aims to generalize the supervised model trained on a source domain to an unlabeled target domain. Marginal distribution alignment of feature spaces is widely used to reduce the domain discr…
ClassificationDomain AdaptationGeneral Classificationimage-classification+2CUDA: Contradistinguisher for Unsupervised Domain Adaptation
In this paper, we propose a simple model referred as Contradistinguisher (CTDR) for unsupervised domain adaptation whose objective is to jointly learn to contradistinguish on unlabeled target domain in a fully unsupervis…
Domain Adaptationdomain classificationUnsupervised Domain Adaptation