State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes
We formulate approximate Bayesian inference in non-conjugate temporal and spatio-temporal Gaussian process models as a simple parameter update rule applied during Kalman smoothing. This viewpoint encompasses most inference schemes, including expectation propagation (EP), the classical (Extended, Unscented, etc.) Kalman smoothers, and variational inference. We provide a unifying perspective on these algorithms, showing how replacing the power EP moment matching step with linearisation recovers the classical smoothers. EP provides some benefits over the traditional methods via introduction of the so-called cavity distribution, and we combine these benefits with the computational efficiency of linearisation, providing extensive empirical analysis demonstrating the efficacy of various algorithms under this unifying framework. We provide a fast implementation of all methods in JAX.
Code (1)
Tasks
Bayesian InferenceComputational EfficiencyGaussian ProcessesVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
State Space Gaussian Processes with Non-Gaussian Likelihood
We provide a comprehensive overview and tooling for GP modeling with non-Gaussian likelihoods using state space methods. The state space formulation allows for solving one-dimensional GP models in $\mathcal{O}(n)$ time a…
Gaussian ProcessesregressionBayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees
We formulate natural gradient variational inference (VI), expectation propagation (EP), and posterior linearisation (PL) as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution…
Bayesian InferenceGaussian ProcessesState Space Modelsvalid+1Bayesian inference for spatio-temporal spike-and-slab priors
In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint. We generalize the spike-and-slab prior distribution to encode a priori correlation of …
Bayesian InferenceExpectation Propagation
Variational inference is a powerful concept that underlies many iterative approximation algorithms; expectation propagation, mean-field methods and belief propagations were all central themes at the school that can be pe…
Variational InferenceOn numerical approximation schemes for expectation propagation
Several numerical approximation strategies for the expectation-propagation algorithm are studied in the context of large-scale learning: the Laplace method, a faster variant of it, Gaussian quadrature, and a deterministi…