Development of Amari Alpha Divergence-Based Gradient-Descent Least Mean Square Algorithm
In this brief, a new least mean square (LMS) adaptive filtering algorithm based on Amari-Alpha information theoretic divergence is proposed which is named as Amari-Alpha LMS (AALMS). The local convergence and stability analysis show the upper bound of step size for tractable analysis. The steady-state performance of the proposed algorithm is analyzed, and the mean-square deviation (MSD) at steady-state is derived. Error-in-variable (EIV) model where input and desired signals both are corrupted with Gaussian noise is considered in this brief. Simulation results and computational complexity analysis show that the proposed AALMS algorithm performs better in comparison to well-known algorithms in terms of MSD in stationary and non-stationary scenarios. Comparison of different distance measures that can be derived from Amari-Alpha divergence is also carried out in terms of the MSD.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Information Geometry and Classical Cramér-Rao Type Inequalities
We examine the role of information geometry in the context of classical Cram\'er-Rao (CR) type inequalities. In particular, we focus on Eguchi's theory of obtaining dualistic geometric structures from a divergence functi…
Vocal Bursts Type PredictionBlack-box $α$-divergence Minimization
Black-box alpha (BB-$\alpha$) is a new approximate inference method based on the minimization of $\alpha$-divergences. BB-$\alpha$ scales to large datasets because it can be implemented using stochastic gradient descent.…
General ClassificationregressionThe Information Geometry of Mirror Descent
Information geometry applies concepts in differential geometry to probability and statistics and is especially useful for parameter estimation in exponential families where parameters are known to lie on a Riemannian man…
parameter estimationMonotonic Alpha-divergence Minimisation for Variational Inference
In this paper, we introduce a novel family of iterative algorithms which carry out $\alpha$-divergence minimisation in a Variational Inference context. They do so by ensuring a systematic decrease at each step in the $\a…
Variational InferenceMixture weights optimisation for Alpha-Divergence Variational Inference
This paper focuses on $\alpha$-divergence minimisation methods for Variational Inference. More precisely, we are interested in algorithms optimising the mixture weights of any given mixture model, without any information…
Variational Inference