Sequential Gibbs Posteriors with Applications to Principal Component Analysis
Gibbs posteriors are proportional to a prior distribution multiplied by an exponentiated loss function, with a key tuning parameter weighting information in the loss relative to the prior and providing a control of posterior uncertainty. Gibbs posteriors provide a principled framework for likelihood-free Bayesian inference, but in many situations, including a single tuning parameter inevitably leads to poor uncertainty quantification. In particular, regardless of the value of the parameter, credible regions have far from the nominal frequentist coverage even in large samples. We propose a sequential extension to Gibbs posteriors to address this problem. We prove the proposed sequential posterior exhibits concentration and a Bernstein-von Mises theorem, which holds under easy to verify conditions in Euclidean space and on manifolds. As a byproduct, we obtain the first Bernstein-von Mises theorem for traditional likelihood-based Bayesian posteriors on manifolds. All methods are illustrated with an application to principal component analysis.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceUncertainty QuantificationSimilar Papers 제목 키워드 기반
Principal component analysis for Gaussian process posteriors
This paper proposes an extension of principal component analysis for Gaussian process (GP) posteriors, denoted by GP-PCA. Since GP-PCA estimates a low-dimensional space of GP posteriors, it can be used for meta-learning,…
Meta-LearningVariational InferenceWhen Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating
Loss-based updating, including generalized Bayes, Gibbs, and quasi-posteriors, replaces likelihoods by a user-chosen loss and produces a posterior-like distribution via exponential tilt. We give a decision-theoretic char…
Practical calibration of the temperature parameter in Gibbs posteriors
PAC-Bayesian algorithms and Gibbs posteriors are gaining popularity due to their robustness against model misspecification even when Bayesian inference is inconsistent. The PAC-Bayesian alpha-posterior is a generalizatio…
Bayesian InferencePAC-Bayes Bounds for Gibbs Posteriors via Singular Learning Theory
We derive explicit non-asymptotic PAC-Bayes generalization bounds for Gibbs posteriors, that is, data-dependent distributions over model parameters obtained by exponentially tilting a prior with the empirical risk. Unlik…
Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inference
This paper considers a non-standard problem of generating samples from a low-temperature Gibbs distribution with \emph{constrained} support, when some of the coordinates of the mode lie on the boundary. These coordinates…
Bayesian Inference