An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift
A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an approach for semi-supervised learning algorithms that is capable of addressing this issue. Our framework also recovers some popular methods, including entropy minimization and pseudo-labeling. We provide new information-theoretical based generalization error upper bounds inspired by our novel framework. Our bounds are applicable to both general semi-supervised learning and the covariate-shift scenario. Finally, we show numerically that our method outperforms previous approaches proposed for semi-supervised learning under the covariate shift.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Semi-Supervised Kernel Two-Sample Test
We consider the problem of two-sample testing in a semi-supervised setting with abundant unlabeled covariate data. Standard two-sample tests neglect covariate information, which has the potential to significantly boost p…
Two-sample testingAdaptive Semi-Supervised Inference for Optimal Treatment Decisions with Electronic Medical Record Data
A treatment regime is a rule that assigns a treatment to patients based on their covariate information. Recently, estimation of the optimal treatment regime that yields the greatest overall expected clinical outcome of i…
ImputationSufficient Dimension Reduction for Average Causal Effect Estimation
Having a large number of covariates can have a negative impact on the quality of causal effect estimation since confounding adjustment becomes unreliable when the number of covariates is large relative to the samples ava…
counterfactualDimensionality ReductionOptimal Policy Adaptation under Covariate Shift
Transfer learning of prediction models has been extensively studied, while the corresponding policy learning approaches are rarely discussed. In this paper, we propose principled approaches for learning the optimal polic…
Transfer LearningSemi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce. We study semi-supervised regression in this setting, and propose a…