Adversarial Distillation of Bayesian Neural Network Posteriors

2018-06-27 · Kuan-Chieh Wang, Paul Vicol, James Lucas, Li Gu, Roger Grosse, Richard Zemel

Bayesian neural networks (BNNs) allow us to reason about uncertainty in a principled way. Stochastic Gradient Langevin Dynamics (SGLD) enables efficient BNN learning by drawing samples from the BNN posterior using mini-batches. However, SGLD and its extensions require storage of many copies of the model parameters, a potentially prohibitive cost, especially for large neural networks. We propose a framework, Adversarial Posterior Distillation, to distill the SGLD samples using a Generative Adversarial Network (GAN). At test-time, samples are generated by the GAN. We show that this distillation framework incurs no loss in performance on recent BNN applications including anomaly detection, active learning, and defense against adversarial attacks. By construction, our framework not only distills the Bayesian predictive distribution, but the posterior itself. This allows one to compute quantities such as the approximate model variance, which is useful in downstream tasks. To our knowledge, these are the first results applying MCMC-based BNNs to the aforementioned downstream applications.

📄 PDF Abstract BibTeX arXiv:1806.10317

Code (1)

wangkua1/apd_public 공식 구현 pytorch

Tasks

Active Learning Anomaly DetectionGenerative Adversarial Network

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dogecoin Customer Service Number +1-833-534-1729 설명 없음

Adversarial Bayesian Simulation

2022-08-25 · Yuexi Wang, Veronika Ročková

In the absence of explicit or tractable likelihoods, Bayesians often resort to approximate Bayesian computation (ABC) for inference. Our work bridges ABC with deep neural implicit samplers based on generative adversarial…

The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks

2021-01-07 · Arno Blaas, Stephen J. Roberts

It is desirable, and often a necessity, for machine learning models to be robust against adversarial attacks. This is particularly true for Bayesian models, as they are well-suited for safety-critical applications, in wh…

Adversarial Robustness Variational Inference

Generalization Certificates for Adversarially Robust Bayesian Linear Regression

2025-02-20 · Mahalakshmi Sabanayagam, Russell Tsuchida, Cheng Soon Ong, Debarghya Ghoshdastidar

Adversarial robustness of machine learning models is critical to ensuring reliable performance under data perturbations. Recent progress has been on point estimators, and this paper considers distributional predictors. F…

Adversarial Robustness Bayesian Inference regression

PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysis

2026-06-04 · Ziling Liang, Xinping Yi, Qingsong Wen, Shi Jin arxiv

Whilst the vulnerability of graph neural networks (GNNs) to adversarial attacks poses a critical threat to graph representation learning, the understanding of the robust generalization behavior remains a fundamental chal…

Graph Representation Learning Adversarial Robustness Graph Classification

Practical calibration of the temperature parameter in Gibbs posteriors

2020-04-22 · Lucie Perrotta

PAC-Bayesian algorithms and Gibbs posteriors are gaining popularity due to their robustness against model misspecification even when Bayesian inference is inconsistent. The PAC-Bayesian alpha-posterior is a generalizatio…

Bayesian Inference