Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC
We present a framework for approximate Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained due to computational constraints, which is becoming increasingly common for applications of complex models. We model the log-likelihood function using a Gaussian process (GP) and the main methodological innovation is to apply this model to emulate the progression that an exact Metropolis-Hastings (MH) sampler would take if it was applicable. Informative log-likelihood evaluation locations are selected using a sequential experimental design strategy until the MH accept/reject decision is done accurately enough according to the GP model. The resulting approximate sampler is conceptually simple and sample-efficient. It is also more robust to violations of GP modelling assumptions compared with earlier, related "Bayesian optimisation-like" methods tailored for Bayesian inference. We discuss some theoretical aspects and various interpretations of the resulting approximate MH sampler, and demonstrate its benefits in the context of Bayesian and generalised Bayesian likelihood-free inference for simulator-based statistical models.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceBayesian OptimisationExperimental DesignMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational Bayesian Monte Carlo with Noisy Likelihoods
Variational Bayesian Monte Carlo (VBMC) is a recently introduced framework that uses Gaussian process surrogates to perform approximate Bayesian inference in models with black-box, non-cheap likelihoods. In this work, we…
Bayesian InferenceBayesian optimisation for fast approximate inference in state-space models with intractable likelihoods
We consider the problem of approximate Bayesian parameter inference in non-linear state-space models with intractable likelihoods. Sequential Monte Carlo with approximate Bayesian computations (SMC-ABC) is one approach t…
Bayesian OptimisationState Space ModelsFast post-process Bayesian inference with Variational Sparse Bayesian Quadrature
In applied Bayesian inference scenarios, users may have access to a large number of pre-existing model evaluations, for example from maximum-a-posteriori (MAP) optimization runs. However, traditional approximate inferenc…
Active LearningBayesian InferenceVariational InferenceVariational Bayesian Monte Carlo
Many probabilistic models of interest in scientific computing and machine learning have expensive, black-box likelihoods that prevent the application of standard techniques for Bayesian inference, such as MCMC, which wou…
Bayesian InferenceModel SelectionVariational InferencePseudo-Bayesian Learning via Direct Loss Minimization with Applications to Sparse Gaussian Process Models
We propose that approximate Bayesian algorithms should optimize a new criterion, directly derived from the loss, to calculate their approximate posterior which we refer to as pseudo-posterior. Unlike standard variational…
Variational Inference