Robustness Guarantees for Bayesian Inference with Gaussian Processes
Bayesian inference and Gaussian processes are widely used in applications ranging from robotics and control to biological systems. Many of these applications are safety-critical and require a characterization of the uncertainty associated with the learning model and formal guarantees on its predictions. In this paper we define a robustness measure for Bayesian inference against input perturbations, given by the probability that, for a test point and a compact set in the input space containing the test point, the prediction of the learning model will remain $\delta-$close for all the points in the set, for $\delta>0.$ Such measures can be used to provide formal guarantees for the absence of adversarial examples. By employing the theory of Gaussian processes, we derive tight upper bounds on the resulting robustness by utilising the Borell-TIS inequality, and propose algorithms for their computation. We evaluate our techniques on two examples, a GP regression problem and a fully-connected deep neural network, where we rely on weak convergence to GPs to study adversarial examples on the MNIST dataset.
Code (1)
Tasks
Bayesian InferenceGaussian ProcessesSimilar Papers 제목 키워드 기반
Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees
Gaussian processes (GPs) offer a flexible class of priors for nonparametric Bayesian regression, but popular GP posterior inference methods are typically prohibitively slow or lack desirable finite-data guarantees on qua…
Gaussian ProcessesregressionVariational InferenceLearning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
Gaussian Processes (GPs) are a generic modelling tool for supervised learning. While they have been successfully applied on large datasets, their use in safety-critical applications is hindered by the lack of good perfor…
Gaussian ProcessesGeneralization BoundsregressionBayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees
We formulate natural gradient variational inference (VI), expectation propagation (EP), and posterior linearisation (PL) as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution…
Bayesian InferenceGaussian ProcessesState Space Modelsvalid+1Fast Bayesian Inference for Gaussian Cox Processes via Path Integral Formulation
Gaussian Cox processes are widely-used point process models that use a Gaussian process to describe the Bayesian a priori uncertainty present in latent intensity functions. In this paper, we propose a novel Bayesian infe…
Bayesian InferenceRobust Gaussian Processes via Relevance Pursuit
Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. However, standard GP models assume homosk…
Bayesian OptimizationGaussian Processesregression