Variance-reduced Domain Adaptation using Paired Sampling
Correlation alignment and the maximum mean discrepancy are two widely used distribution-matching frameworks for unsupervised domain adaptation (UDA). However, high variance in these losses has been shown to undermine their effectiveness in minibatch optimisation settings. Furthermore, the losses lack finite-sum structure, which renders them incompatible with classical stochastic variance reduction (SVR) methods. This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives. PSDA pairs observations both within and across domains, to form quadruplets that are always sampled together during training. The pairings are designed to minimise expected gradient variance, and reduce to solving a set of linear assignment problems. Our simulations demonstrate reduced variance compared to related methods, and experiments on three domain shift datasets show improved target domain accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Unsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Variance Matters: Improving Domain Adaptation via Stratified Sampling
Domain shift remains a key challenge in deploying machine learning models to the real world. Unsupervised domain adaptation (UDA) aims to address this by minimising domain discrepancy during training, but the discrepancy…
Unsupervised Domain AdaptationOrder Matters: Improving Domain Adaptation by Reordering Data
Domain shift remains a key challenge in deploying machine learning models to the real world. Unsupervised domain adaptation (UDA) aims to address this by minimising domain discrepancy during training, but the discrepancy…
Unsupervised Domain AdaptationImage ClassificationStochastic Variance-Reduced Policy Gradient
In this paper, we propose a novel reinforcement- learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs). Stochastic variance-reduced gradien…
Reinforcement LearningNonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs
Neural audio super-resolution models are typically trained on low- and high-resolution audio signal pairs. Although these methods achieve highly accurate super-resolution if the acoustic characteristics of the input data…
Audio Super-ResolutionDomain AdaptationSuper-ResolutionX4Val: Learning Neural Surrogates for Variance-Reduced Policy Evaluation
Rigorous evaluation of learning-based robotic systems is an essential prerequisite for deployment. However, real-world test data is expensive to gather; moreover, in a typical iterative development context, data gathered…
Robot ManipulationAutonomous Driving