Validated Variational Inference via Practical Posterior Error Bounds
Variational inference has become an increasingly attractive fast alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, a major obstacle to the widespread use of variational methods is the lack of post-hoc accuracy measures that are both theoretically justified and computationally efficient. In this paper, we provide rigorous bounds on the error of posterior mean and uncertainty estimates that arise from full-distribution approximations, as in variational inference. Our bounds are widely applicable, as they require only that the approximating and exact posteriors have polynomial moments. Our bounds are also computationally efficient for variational inference because they require only standard values from variational objectives, straightforward analytic calculations, and simple Monte Carlo estimates. We show that our analysis naturally leads to a new and improved workflow for validated variational inference. Finally, we demonstrate the utility of our proposed workflow and error bounds on a robust regression problem and on a real-data example with a widely used multilevel hierarchical model.
Code (1)
Tasks
Bayesian InferenceVariational InferenceSimilar Papers 제목 키워드 기반
Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection
Variational inference has recently emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) in large-scale Bayesian inference. The core idea is to trade statistical accuracy for computational eff…
Bayesian InferenceComputational EfficiencyModel SelectionStochastic Optimization+2Robust and Scalable Variational Bayes
We propose a robust and scalable framework for variational Bayes (VB) that effectively handles outliers and contamination of arbitrary nature in large datasets. Our approach divides the dataset into disjoint subsets, com…
Bayesian InferenceStacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
When working with multimodal Bayesian posterior distributions, Markov chain Monte Carlo (MCMC) algorithms have difficulty moving between modes, and default variational or mode-based approximate inferences will understate…
Bayesian InferenceregressionVariable SelectionVariational InferenceGaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders
Variational autoencoder (VAE) is a very successful generative model whose key element is the so called amortized inference network, which can perform test time inference using a single feed forward pass. Unfortunatel…
Gaussian ProcessesScore Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoising
Real-world single image denoising is crucial and practical in computer vision. Bayesian inversions combined with score priors now have proven effective for single image denoising but are limited to white Gaussian noise. …
DenoisingImage DenoisingVariational Inference