On Regularization Parameter Estimation under Covariate Shift
This paper identifies a problem with the usual procedure for L2-regularization parameter estimation in a domain adaptation setting. In such a setting, there are differences between the distributions generating the training data (source domain) and the test data (target domain). The usual cross-validation procedure requires validation data, which can not be obtained from the unlabeled target data. The problem is that if one decides to use source validation data, the regularization parameter is underestimated. One possible solution is to scale the source validation data through importance weighting, but we show that this correction is not sufficient. We conclude the paper with an empirical analysis of the effect of several importance weight estimators on the estimation of the regularization parameter.
Code (1)
Tasks
Domain AdaptationL2 Regularizationparameter estimationSimilar Papers 제목 키워드 기반
PIcsC: Partitioning-Induced Covariate Shift Correction
Covariate shift across training-data partitions biases model selection and parameter estimation in cross-validation, lifelong learning, and federated learning. We propose \textit{Partition-Induced Covariate-shift Correct…
Federated LearningTowards regularized learning from functional data with covariate shift
This paper investigates a general regularization framework for unsupervised domain adaptation in vector-valued regression under the covariate shift assumption, utilizing vector-valued reproducing kernel Hilbert spaces (v…
Unsupervised Domain AdaptationOptimal Ridge Regularization for Out-of-Distribution Prediction
We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-distribution prediction, where the test distribution deviates arbitrarily from the train distribution. We establish general conditio…
PredictionregressionHigh-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization
This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate …
Class Prior Estimation under Covariate Shift: No Problem?
We show that in the context of classification the property of source and target distributions to be related by covariate shift may be lost if the information content captured in the covariates is reduced, for instance by…