On numerical approximation schemes for expectation propagation
Several numerical approximation strategies for the expectation-propagation algorithm are studied in the context of large-scale learning: the Laplace method, a faster variant of it, Gaussian quadrature, and a deterministic version of variational sampling (i.e., combining quadrature with variational approximation). Experiments in training linear binary classifiers show that the expectation-propagation algorithm converges best using variational sampling, while it also converges well using Laplace-style methods with smooth factors but tends to be unstable with non-differentiable ones. Gaussian quadrature yields unstable behavior or convergence to a sub-optimal solution in most experiments.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Conditional Expectation Propagation
Expectation propagation (EP) is a powerful approximate inference algorithm. However, a critical barrier in applying EP is that the moment matching in message updates can be intractable. Handcrafting approximations is usu…
Computational EfficiencyA Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
Gaussian processes (GPs) are flexible distributions over functions that enable high-level assumptions about unknown functions to be encoded in a parsimonious, flexible and general way. Although elegant, the application o…
Gaussian ProcessesBayesian inference for spatio-temporal spike-and-slab priors
In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint. We generalize the spike-and-slab prior distribution to encode a priori correlation of …
Bayesian InferenceExpectation Propagation
Variational inference is a powerful concept that underlies many iterative approximation algorithms; expectation propagation, mean-field methods and belief propagations were all central themes at the school that can be pe…
Variational InferenceBayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees
We formulate natural gradient variational inference (VI), expectation propagation (EP), and posterior linearisation (PL) as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution…
Bayesian InferenceGaussian ProcessesState Space Modelsvalid+1