Stein Variational Gaussian Processes

2020-09-25 · Thomas Pinder, Christopher Nemeth, David Leslie

We show how to use Stein variational gradient descent (SVGD) to carry out inference in Gaussian process (GP) models with non-Gaussian likelihoods and large data volumes. Markov chain Monte Carlo (MCMC) is extremely computationally intensive for these situations, but the parametric assumptions required for efficient variational inference (VI) result in incorrect inference when they encounter the multi-modal posterior distributions that are common for such models. SVGD provides a non-parametric alternative to variational inference which is substantially faster than MCMC. We prove that for GP models with Lipschitz gradients the SVGD algorithm monotonically decreases the Kullback-Leibler divergence from the sampling distribution to the true posterior. Our method is demonstrated on benchmark problems in both regression and classification, a multimodal posterior, and an air quality example with 550,134 spatiotemporal observations, showing substantial performance improvements over MCMC and VI.

📄 PDF Abstract BibTeX arXiv:2009.12141

Code (1)

thomaspinder/SteinGP 공식 구현 tf

Tasks

Gaussian ProcessesVariational Inference

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning

2022-05-12 · Veit D. Wild, Robert Hu, Dino Sejdinovic

We develop a framework for generalized variational inference in infinite-dimensional function spaces and use it to construct a method termed Gaussian Wasserstein inference (GWI). GWI leverages the Wasserstein distance be…

Gaussian Processes Uncertainty Quantification Variational Inference

A variational approach to dimension-free self-normalized concentration

2025-08-08 · Ben Chugg, Aaditya Ramdas arxiv

We study the self-normalized concentration of vector-valued stochastic processes. We focus on bounds for "sub-$ψ$" processes, a well-known and quite general class of process that encompasses a wide variety of well-known …

Functional Variational Bayesian Neural Networks

2019-03-14 · ICLR 2019 5 · Shengyang Sun, Guodong Zhang, Jiaxin Shi, Roger Grosse

Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functio…

Bayesian Inference Gaussian Processes Variational Inference

Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes

2023-09-22 · Jian Xu, Shian Du, Junmei Yang, Qianli Ma 외

Deep Gaussian Process (DGP) models offer a powerful nonparametric approach for Bayesian inference, but exact inference is typically intractable, motivating the use of various approximations. However, existing approaches,…

Bayesian Inference Gaussian Processes Stochastic Optimization Variational Inference

Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space

2023-04-10 · Michael Diao, Krishnakumar Balasubramanian, Sinho Chewi, Adil Salim

Variational inference (VI) seeks to approximate a target distribution $\pi$ by an element of a tractable family of distributions. Of key interest in statistics and machine learning is Gaussian VI, which approximates $\pi…

Variational Inference