On the Use of Sparse Filtering for Covariate Shift Adaptation
In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional distribution of the labels has a structure explained by a cosine metric. To overcome this limitation, we propose a new algorithm, named periodic sparse filtering, and carry out the same theoretical analysis regarding covariate shift adaptation. We show that periodic sparse filtering can perform adaptation under the looser and more realistic requirement that the conditional distribution of the labels has a periodic structure, which may be satisfied, for instance, by user-dependent data sets. We experimentally validate our theoretical results on synthetic data. Moreover, we apply periodic sparse filtering to real-world data sets to demonstrate that this simple and computationally efficient algorithm is able to achieve competitive performances.
Code (1)
Similar Papers 제목 키워드 기반
PAC Learning Guarantees Under Covariate Shift
We consider the Domain Adaptation problem, also known as the covariate shift problem, where the distributions that generate the training and test data differ while retaining the same labeling function. This problem occur…
Domain AdaptationPAC learningTransfer LearningEffective Sample Size, Dimensionality, and Generalization in Covariate Shift Adaptation
In supervised learning, training and test datasets are often sampled from distinct distributions. Domain adaptation techniques are thus required. Covariate shift adaptation yields good generalization performance when dom…
Dimensionality ReductionDomain Adaptationfeature selectionGeneralization BoundsFederated Covariate Shift Adaptation for Missing Target Output Values
The most recent multi-source covariate shift algorithm is an efficient hyperparameter optimization algorithm for missing target output. In this paper, we extend this algorithm to the framework of federated learning. For …
Domain AdaptationFederated LearningHyperparameter OptimizationInformation Geometrically Generalized Covariate Shift Adaptation
Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular, the phenomenon that the marginal distr…
Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment
Unsupervised knowledge transfer has a great potential to improve the generalizability of deep models to novel domains. Yet the current literature assumes that the label distribution is domain-invariant and only aligns th…
Domain AdaptationTransfer Learning