Risk Bounds for Unsupervised Cross-Domain Mapping with IPMs
The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications. This lacuna is especially troubling, given the clear ambiguity in such mappings. We work with adversarial training methods based on IPMs and derive a novel risk bound, which upper bounds the risk between the learned mapping $h$ and the target mapping $y$, by a sum of three terms: (i) the risk between $h$ and the most distant alternative mapping that was learned by the same cross-domain mapping algorithm, (ii) the minimal discrepancy between the target domain and the domain obtained by applying a hypothesis $h^*$ on the samples of the source domain, where $h^*$ is a hypothesis selectable by the same algorithm. The bound is directly related to Occam's razor and encourages the selection of the minimal architecture that supports a small mapping discrepancy and (iii) an approximation error term that decreases as the complexity of the class of discriminators increases and is empirically shown to be small. The bound leads to multiple algorithmic consequences, including a method for hyperparameters selection and for early stopping in cross-domain mapping GANs. We also demonstrate a novel capability for unsupervised learning of estimating confidence in the mapping of every specific sample.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Estimating the Success of Unsupervised Image to Image Translation
While in supervised learning, the validation error is an unbiased estimator of the generalization (test) error and complexity-based generalization bounds are abundant, no such bounds exist for learning a mapping in an un…
Generalization BoundsImage-to-Image TranslationTranslationUnsupervised Image-To-Image TranslationInformation-Theoretic Analysis of Unsupervised Domain Adaptation
This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). We present novel upper bounds for two notions of generalization errors. The first notion measures t…
Domain AdaptationUnsupervised Domain AdaptationA Theory of Output-Side Unsupervised Domain Adaptation
When learning a mapping from an input space to an output space, the assumption that the sample distribution of the training data is the same as that of the test data is often violated. Unsupervised domain shift methods a…
Domain AdaptationGeneralization BoundsUnsupervised Domain AdaptationHow does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?
Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. Classical UDA learning bounds show that target risk is uppe…
Domain AdaptationUnsupervised Domain AdaptationUnifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy
Differentially private (DP) mechanisms are difficult to interpret and calibrate because existing methods for mapping standard privacy parameters to concrete privacy risks -- re-identification, attribute inference, and da…
Text Classification