Local Max-Entropy and Free Energy Principles, Belief Diffusions and their Singularities
A comprehensive picture of three Bethe-Kikuchi variational principles including their relationship to belief propagation (BP) algorithms on hypergraphs is given. The structure of BP equations is generalized to define continuous-time diffusions, solving localized versions of the max-entropy principle (A), the variational free energy principle (B), and a less usual equilibrium free energy principle (C), Legendre dual to A. Both critical points of Bethe-Kikuchi functionals and stationary beliefs are shown to lie at the non-linear intersection of two constraint surfaces, enforcing energy conservation and marginal consistency respectively. The hypersurface of singular beliefs, accross which equilibria become unstable as the constraint surfaces meet tangentially, is described by polynomial equations in the convex polytope of consistent beliefs. This polynomial is expressed by a loop series expansion for graphs of binary variables.
Code (1)
Similar Papers 제목 키워드 기반
Local Max-Entropy and Free Energy Principles Solved by Belief Propagation
A statistical system is classically defined on a set of microstates $E$ by a global energy function $H : E \to \mathbb{R}$, yielding Gibbs probability measures (softmins) $\rho^\beta(H)$ for every inverse temperature $\b…
Free Energy Minimization: A Unified Framework for Modelling, Inference, Learning,and Optimization
The goal of these lecture notes is to review the problem of free energy minimization as a unified framework underlying the definition of maximum entropy modelling, generalized Bayesian inference, learning with latent var…
Bayesian InferenceGraph Zeta Function in the Bethe Free Energy and Loopy Belief Propagation
We propose a new approach to the analysis of Loopy Belief Propagation (LBP) by establishing a formula that connects the Hessian of the Bethe free energy with the edge zeta function. The formula has a number of theoretica…
Regionalized Optimization
We propose a theoretical framework for non redundant reconstruction of a global loss from a collection of local ones under constraints given by a functor; we call this loss the regionalized loss in honor to Yedidia, Free…
Active Inference in Discrete State Spaces from First Principles
We seek to clarify the concept of active inference by disentangling it from the Free Energy Principle. We show how the optimizations that need to be carried out in order to implement active inference in discrete state sp…