PAC-Bayesian Generalization Bounds for Adversarial Generative Models
We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.
Code (1)
Tasks
Dimensionality ReductionGeneralization BoundsSimilar Papers 제목 키워드 기반
PAC-Bayesian Adversarially Robust Generalization Bounds for Graph Neural Network
Graph neural networks (GNNs) have gained popularity for various graph-related tasks. However, similar to deep neural networks, GNNs are also vulnerable to adversarial attacks. Empirical studies have shown that adversaria…
Generalization BoundsGraph Neural NetworkPAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization
Deep neural networks (DNNs) are vulnerable to adversarial attacks. It is found empirically that adversarially robust generalization is crucial in establishing defense algorithms against adversarial attacks. Therefore, it…
Adversarial RobustnessGeneralization BoundsA PAC-Bayes Analysis of Adversarial Robustness
We propose the first general PAC-Bayesian generalization bounds for adversarial robustness, that estimate, at test time, how much a model will be invariant to imperceptible perturbations in the input. Instead of deriving…
Adversarial RobustnessGeneralization BoundsvalidPAC-Bayesian Generalization Bounds for MultiLayer Perceptrons
We study PAC-Bayesian generalization bounds for Multilayer Perceptrons (MLPs) with the cross entropy loss. Above all, we introduce probabilistic explanations for MLPs in two aspects: (i) MLPs formulate a family of Gibbs …
Generalization BoundsVariational InferenceMisclassification excess risk bounds for PAC-Bayesian classification via convexified loss
PAC-Bayesian bounds have proven to be a valuable tool for deriving generalization bounds and for designing new learning algorithms in machine learning. However, it typically focus on providing generalization bounds with …
Generalization Bounds