Avoiding Non-Integrable Beliefs in Expectation Propagation
Expectation Propagation (EP) is a widely used iterative message-passing algorithm that decomposes a global inference problem into multiple local ones. It approximates marginal distributions as `beliefs'' using intermediate functions called `messages''. It has been shown that the stationary points of EP are the same as corresponding constrained Bethe Free Energy (BFE) optimization problem. Therefore, EP is an iterative method of optimizing the constrained BFE. However, the iterative method may fall out of the feasible set of the BFE optimization problem, i.e., the beliefs are not integrable. In most literature, the authors use various methods to keep all the messages integrable. In most Bayesian estimation problems, limiting the messages to be integrable shrinks the actual feasible set. Furthermore, in extreme cases where the factors are not integrable, making the message itself integrable is not enough to have integrable beliefs. In this paper, two EP frameworks are proposed to ensure that EP has integrable beliefs. Both of the methods allows non-integrable messages. We then investigate the signal recovery problem in Generalized Linear Model (GLM) using our proposed methods.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Expectation Particle Belief Propagation
We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient prop…
Neural Expectation Operators
This paper introduces \textbf{Measure Learning}, a paradigm for modeling ambiguity via non-linear expectations. We define Neural Expectation Operators as solutions to Backward Stochastic Differential Equations (BSDEs) wh…
The Accuracy of Job Seekers' Wage Expectations
We study the accuracy of job seekers' wage expectations by comparing subjective beliefs to objective benchmarks using linked administrative and survey data. Our findings show that especially job seekers with low objectiv…
Decision MakingLearning unbelievable probabilities
Loopy belief propagation performs approximate inference on graphical models with loops. One might hope to compensate for the approximation by adjusting model parameters. Learning algorithms for this purpose have been exp…
Computation of conditional expectations with guarantees
Theoretically, the conditional expectation of a square-integrable random variable $Y$ given a $d$-dimensional random vector $X$ can be obtained by minimizing the mean squared distance between $Y$ and $f(X)$ over all Bore…