A MAP approach for $\ell_q$-norm regularized sparse parameter estimation using the EM algorithm
In this paper, Bayesian parameter estimation through the consideration of the Maximum A Posteriori (MAP) criterion is revisited under the prism of the Expectation-Maximization (EM) algorithm. By incorporating a sparsity-promoting penalty term in the cost function of the estimation problem through the use of an appropriate prior distribution, we show how the EM algorithm can be used to efficiently solve the corresponding optimization problem. To this end, we rely on variance-mean Gaussian mixtures (VMGM) to describe the prior distribution, while we incorporate many nice features of these mixtures to our estimation problem. The corresponding MAP estimation problem is completely expressed in terms of the EM algorithm, which allows for handling nonlinearities and hidden variables that cannot be easily handled with traditional methods. For comparison purposes, we also develop a Coordinate Descent algorithm for the $\ell_q$-norm penalized problem and present the performance results via simulations.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter estimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Infinite-Dimensional Sparse Learning in Linear System Identification
Regularized methods have been widely applied to system identification problems without known model structures. This paper proposes an infinite-dimensional sparse learning algorithm based on atomic norm regularization. At…
Sparse LearningSparse Learning and Class Probability Estimation with Weighted Support Vector Machines
Classification and probability estimation have broad applications in modern machine learning and data science applications, including biology, medicine, engineering, and computer science. The recent development of a clas…
Ensemble LearningSparse LearningVariable SelectionA variational Bayes framework for sparse adaptive estimation
Recently, a number of mostly $\ell_1$-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a B…
Regularized Estimation and Feature Selection in Mixtures of Gaussian-Gated Experts Models
Mixtures-of-Experts models and their maximum likelihood estimation (MLE) via the EM algorithm have been thoroughly studied in the statistics and machine learning literature. They are subject of a growing investigation in…
Clusteringfeature selectionparameter estimationregressionRegularized Maximum Likelihood Estimation and Feature Selection in Mixtures-of-Experts Models
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation …
Clusteringfeature selectionMixture-of-Expertsparameter estimation+1