2-D Cluster Variation Method Free Energy: Fundamentals and Pragmatics
Despite being invented in 1951 by R. Kikuchi, the 2-D Cluster Variation Method (CVM), has not yet received attention. Nevertheless, this method can usefully characterize 2-D topographies using just two parameters; the activation enthalpy and the interaction enthalpy. This Technical Report presents 2-D CVM details, including the dependence of the various configuration variables on the enthalpy parameters, as well as illustrations of various topographies (ranging from scale-free-like to rich club-like) that result from different parameter selection. The complete derivation for the analytic solution, originally presented simply as a result in Kikuchi and Brush (1967) is given here, along with careful comparison of the analytically-predicted configuration variables versus those obtained when performing computational free energy minimization on a 2-D grid. The 2-D CVM can potentially function as a secondary free energy minimization within the hidden layer of a neural network, providing a basis for extending node activations over time and allowing temporal correlation of patterns.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
$k$-means as a variational EM approximation of Gaussian mixture models
We show that $k$-means (Lloyd's algorithm) is obtained as a special case when truncated variational EM approximations are applied to Gaussian Mixture Models (GMM) with isotropic Gaussians. In contrast to the standard way…
ClusteringLoss as the Inconsistency of a Probabilistic Dependency Graph: Choose Your Model, Not Your Loss Function
In a world blessed with a great diversity of loss functions, we argue that that choice between them is not a matter of taste or pragmatics, but of model. Probabilistic depencency graphs (PDGs) are probabilistic models th…
DiversityVariational InferenceFree Energy Minimization Using the 2-D Cluster Variation Method: Initial Code Verification and Validation
A new approach for general artificial intelligence (GAI), building on neural network deep learning architectures, can make use of one or more hidden layers that have the ability to continuously reach a free energy minimu…
A Variational Approach to Parameter Estimation for Characterizing 2-D Cluster Variation Method Topographies
One of the biggest challenges in characterizing 2-D topographies is succinctly communicating the dominant nature of local configurations. In a 2-D grid composed of bistate units, this could be expressed as finding the ch…
parameter estimationTripletVariational InferenceDerivation of the Variational Bayes Equations
The derivation of key equations for the variational Bayes approach is well-known in certain circles. However, translating the fundamental derivations (e.g., as found in Beal's work) to Friston's notation is somewhat deli…