Sufficient Statistic Memory Approximate Message Passing

2022-06-23 · Lei Liu, Shunqi Huang, Brian M. Kurkoski

Approximate message passing (AMP) type algorithms have been widely used in the signal reconstruction of certain large random linear systems. A key feature of the AMP-type algorithms is that their dynamics can be correctly described by state evolution. However, state evolution does not necessarily guarantee the convergence of iterative algorithms. To solve the convergence problem of AMP-type algorithms in principle, this paper proposes a memory AMP (MAMP) under a sufficient statistic condition, named sufficient statistic MAMP (SS-MAMP). We show that the covariance matrices of SS-MAMP are L-banded and convergent. Given an arbitrary MAMP, we can construct the SS-MAMP by damping, which not only ensures the convergence, but also preserves the orthogonality, i.e., its dynamics can be correctly described by state evolution.

📄 PDF Abstract BibTeX arXiv:2206.11674

Code (0)

등록된 구현이 없습니다.

Tasks

Vocal Bursts Type Prediction

Methods 이 논문이 사용한 방법론

AMP Based on the understanding that the flat local minima of the empirical risk cause the model to generalize better. Adversarial Model Perturbation (AMP) improves generalization via…

Sufficient-Statistic Memory AMP

2021-12-31 · Lei Liu, Shunqi Huang, Yuzhi Yang, Zhaoyang Zhang 외

How the brain might work: statistics flowing in redundant population codes

2017-05-15

It is widely believed that the brain performs approximate probabilistic inference to estimate causal variables in the world from ambiguous sensory data. To understand these computations, we need to analyze how informatio…

Replicated Vector Approximate Message Passing For Resampling Problem

2019-05-23 · Takashi Takahashi, Yoshiyuki Kabashima

Resampling techniques are widely used in statistical inference and ensemble learning, in which estimators' statistical properties are essential. However, existing methods are computationally demanding, because repetition…

Ensemble Learning Variable Selection

The Inverse G-Wishart Distribution and Variational Message Passing

2020-05-20 · L. Maestrini, M. P. Wand

Message passing on a factor graph is a powerful paradigm for the coding of approximate inference algorithms for arbitrarily graphical large models. The notion of a factor graph fragment allows for compartmentalization of…

BIG-bench Machine Learning

Properties of Bethe Free Energies and Message Passing in Gaussian Models

2014-01-16 · Botond Cseke, Tom Heskes

We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. We define the Gaussian fractional Bethe free energy in terms of the mome…