PAC-Bayes and Domain Adaptation
We provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in Germain et al. (2013), which relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter domain adaptation bound for the target risk. While this bound stands in the spirit of common domain adaptation works, we derive a second bound (introduced in Germain et al., 2016) that brings a new perspective on domain adaptation by deriving an upper bound on the target risk where the distributions' divergence-expressed as a ratio-controls the trade-off between a source error measure and the target voters' disagreement. We discuss and compare both results, from which we obtain PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian specialization to linear classifiers, we infer two learning algorithms, and we evaluate them on real data.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationGeneralization BoundsSimilar Papers 제목 키워드 기반
PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers
In this paper, we provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different target distributio…
Domain AdaptationPAC-Bayesian Domain Adaptation Bounds for Multi-view learning
This paper presents a series of new results for domain adaptation in the multi-view learning setting. The incorporation of multiple views in the domain adaptation was paid little attention in the previous studies. In thi…
Domain AdaptationGeneralization BoundsMULTI-VIEW LEARNINGDomain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation
Previous studies have shown that leveraging domain index can significantly boost domain adaptation performance (arXiv:2007.01807, arXiv:2202.03628). However, such domain indices are not always available. To address this …
Domain AdaptationAn Improvement to the Domain Adaptation Bound in a PAC-Bayesian context
This paper provides a theoretical analysis of domain adaptation based on the PAC-Bayesian theory. We propose an improvement of the previous domain adaptation bound obtained by Germain et al. in two ways. We first give an…
Domain AdaptationVariational Model Perturbation for Source-Free Domain Adaptation
We aim for source-free domain adaptation, where the task is to deploy a model pre-trained on source domains to target domains. The challenges stem from the distribution shift from the source to the target domain, coupled…
Bayesian InferenceDomain AdaptationmodelSource-Free Domain Adaptation