Support and Invertibility in Domain-Invariant Representations
Learning domain-invariant representations has become a popular approach to unsupervised domain adaptation and is often justified by invoking a particular suite of theoretical results. We argue that there are two significant flaws in such arguments. First, the results in question hold only for a fixed representation and do not account for information lost in non-invertible transformations. Second, domain invariance is often a far too strict requirement and does not always lead to consistent estimation, even under strong and favorable assumptions. In this work, we give generalization bounds for unsupervised domain adaptation that hold for any representation function by acknowledging the cost of non-invertibility. In addition, we show that penalizing distance between densities is often wasteful and propose a bound based on measuring the extent to which the support of the source domain covers the target domain. We perform experiments on well-known benchmarks that illustrate the short-comings of current standard practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationGeneralization BoundsUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Production Function Estimation without Invertibility: Imperfectly Competitive Environments and Demand Shocks
We advance the proxy variable approach to production function estimation. We show that the invertibility assumption at its heart is testable. We characterize what goes wrong if invertibility fails and what can still be d…
Learning Invariant Representations with a Nonparametric Nadaraya-Watson Head
Machine learning models will often fail when deployed in an environment with a data distribution that is different than the training distribution. When multiple environments are available during training, many methods ex…
Domain GeneralizationNeural Networks Preserve Invertibility Across Iterations: A Possible Source of Implicit Data Augmentation
Determining what kind of representations neural networks learn, and how this may relate to generalization, remains a challenging problem. Previous work has utilized a rich set of methods to invert layer representations o…
Data AugmentationTowards Practical Non-Adversarial Distribution Matching
Distribution matching can be used to learn invariant representations with applications in fairness and robustness. Most prior works resort to adversarial matching methods but the resulting minimax problems are unstable a…
FairnessRepresentation LearningDomain-Invariant Representations: A Look on Compression and Weights
Learning Invariant Representations to adapt deep classifiers of a source domain to a new target domain has recently attracted much attention. In this paper, we show that the search for invariance favors the compression …
Domain Adaptation