Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference
We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a \textit{self-consistency loss} which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation.
Code (1)
Tasks
Bayesian InferenceSimilar Papers 제목 키워드 기반
Improving the Accuracy of Amortized Model Comparison with Self-Consistency
Amortized Bayesian model comparison (BMC) enables fast probabilistic ranking of models via simulation-based training of neural surrogates. However, the accuracy of neural surrogates deteriorates when simulation models ar…
Improving the Accuracy of Amortized Model Comparison with Self-Consistency
Amortized Bayesian inference (ABI) offers fast, scalable approximations to posterior densities by training neural surrogates on data simulated from the statistical model. However, ABI methods are highly sensitive to mode…
Bayesian InferenceRobust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data
Amortized Bayesian inference (ABI) with neural networks can solve probabilistic inverse problems orders of magnitude faster than classical methods. However, ABI is not yet sufficiently robust for widespread and safe appl…
Bayesian InferenceUnsupervised Continual Learning for Amortized Bayesian Inference
Amortized Bayesian Inference (ABI) enables efficient posterior estimation using generative neural networks trained on simulated data, but often suffers from performance degradation under model misspecification. While sel…
Continual LearningBayesian InferencePosition: The Future of Bayesian Prediction Is Prior-Fitted
Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods de…
Bayesian InferencePosition