Understanding and Estimating the Adaptability of Domain-Invariant Representations
Learning domain-invariant representations is a popular approach to unsupervised domain adaptation, i.e., generalizing from a source domain with labels to an unlabeled target domain. In this work, we aim to better understand and estimate the effect of domain-invariant representations on generalization to the target. In particular, we study the effect of the complexity of the latent, domain-invariant representation, and find that it has a significant influence on the target risk. Based on these findings, we propose a general approach for addressing this complexity tradeoff in neural networks. We also propose a method for estimating how well a model based on domain-invariant representations will perform on the target domain, without having seen any target labels. Applications of our results include model selection, deciding early stopping, and predicting the adaptability of a model between domains.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationModel SelectionUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Domain-Invariant Representations: A Look on Compression and Weights
Learning Invariant Representations to adapt deep classifiers of a source domain to a new target domain has recently attracted much attention. In this paper, we show that the search for invariance favors the compression …
Domain AdaptationTowards Category and Domain Alignment: Category-Invariant Feature Enhancement for Adversarial Domain Adaptation
Adversarial domain adaptation has made impressive advances in transferring knowledge from the source domain to the target domain by aligning feature distributions of both domains. These methods focus on minimizing domain…
Domain AdaptationTransfer Learning with Label Noise
Transfer learning aims to improve learning in target domain by borrowing knowledge from a related but different source domain. To reduce the distribution shift between source and target domains, recent methods have focus…
DenoisingTransfer LearningEstimating Generalization under Distribution Shifts via Domain-Invariant Representations
When machine learning models are deployed on a test distribution different from the training distribution, they can perform poorly, but overestimate their performance. In this work, we aim to better estimate a model's pe…
Domain AdaptationModel SelectionTAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification
Robust domain adaptation against adversarial attacks is a critical research area that aims to develop models capable of maintaining consistent performance across diverse and challenging domains. In this paper, we derive …
Domain AdaptationDomain Generalization