Information-Theoretic Analysis of Unsupervised Domain Adaptation
This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). We present novel upper bounds for two notions of generalization errors. The first notion measures the gap between the population risk in the target domain and that in the source domain, and the second measures the gap between the population risk in the target domain and the empirical risk in the source domain. While our bounds for the first kind of error are in line with the traditional analysis and give similar insights, our bounds on the second kind of error are algorithm-dependent, which also provide insights into algorithm designs. Specifically, we present two simple techniques for improving generalization in UDA and validate them experimentally.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Domain Adaptation via Maximizing Surrogate Mutual Information
Unsupervised domain adaptation (UDA) aims to predict unlabeled data from target domain with access to labeled data from the source domain. In this work, we propose a novel framework called SIDA (Surrogate Mutual Informat…
Domain AdaptationTransfer LearningUnsupervised Domain AdaptationUnsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain. However, in many real-world settings one seeks to adapt to multiple, …
DisentanglementDomain AdaptationMulti-target Domain AdaptationUnsupervised Domain AdaptationOn Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation
Due to privacy, storage, and other constraints, there is a growing need for unsupervised domain adaptation techniques in machine learning that do not require access to the data used to train a collection of source models…
Domain AdaptationSource-Free Domain AdaptationUnsupervised Domain AdaptationRethinking Propagation for Unsupervised Graph Domain Adaptation
Unsupervised Graph Domain Adaptation (UGDA) aims to transfer knowledge from a labelled source graph to an unlabelled target graph in order to address the distribution shifts between graph domains. Previous works have pri…
Domain AdaptationGRAPH DOMAIN ADAPTATIONTask Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning
Unsupervised reinforcement learning (URL) aims to learn general skills for unseen downstream tasks. Mutual Information Skill Learning (MISL) addresses URL by maximizing the mutual information between states and skills bu…
DisentanglementDiversityUnsupervised Reinforcement Learning