Expectation Consistent Approximate Inference: Generalizations and Convergence
Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Noisy Expectation-Maximization: Applications and Generalizations
We present a noise-injected version of the Expectation-Maximization (EM) algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM algorithm uses noise to speed up the convergence of the EM algorithm. The NE…
Approximate message-passing for convex optimization with non-separable penalties
We introduce an iterative optimization scheme for convex objectives consisting of a linear loss and a non-separable penalty, based on the expectation-consistent approximation and the vector approximate message-passing (V…
Online Bootstrap Inference with Nonconvex Stochastic Gradient Descent Estimator
In this paper, we investigate the theoretical properties of stochastic gradient descent (SGD) for statistical inference in the context of nonconvex optimization problems, which have been relatively unexplored compared to…
validA Stochastic Path Integral Differential EstimatoR Expectation Maximization Algorithm
The Expectation Maximization (EM) algorithm is of key importance for inference in latent variable models including mixture of regressors and experts, missing observations. This paper introduces a novel EM algorithm, call…
Adaptive importance sampling for heavy-tailed distributions via $α$-divergence minimization
Adaptive importance sampling (AIS) algorithms are widely used to approximate expectations with respect to complicated target probability distributions. When the target has heavy tails, existing AIS algorithms can provide…
Bayesian OptimizationVariational Inference