Sparsity via Hyperpriors: A Theoretical and Algorithmic Study under Empirical Bayes Framework
This paper presents a comprehensive analysis of hyperparameter estimation within the empirical Bayes framework (EBF) for sparse learning. By studying the influence of hyperpriors on the solution of EBF, we establish a theoretical connection between the choice of the hyperprior and the sparsity as well as the local optimality of the resulting solutions. We show that some strictly increasing hyperpriors, such as half-Laplace and half-generalized Gaussian with the power in $(0,1)$, effectively promote sparsity and improve solution stability with respect to measurement noise. Based on this analysis, we adopt a proximal alternating linearized minimization (PALM) algorithm with convergence guaranties for both convex and concave hyperpriors. Extensive numerical tests on two-dimensional image deblurring problems demonstrate that introducing appropriate hyperpriors significantly promotes the sparsity of the solution and enhances restoration accuracy. Furthermore, we illustrate the influence of the noise level and the ill-posedness of inverse problems to EBF solutions.
Code (0)
등록된 구현이 없습니다.
Tasks
Image DeblurringSparse LearningSimilar Papers 제목 키워드 기반
Sparse Bayesian Lasso via a Variable-Coefficient $\ell_1$ Penalty
Modern statistical learning algorithms are capable of amazing flexibility, but struggle with interpretability. One possible solution is sparsity: making inference such that many of the parameters are estimated as being i…
regressionUncertainty QuantificationAlgorithmic Aspects of Inverse Problems Using Generative Models
The traditional approach of hand-crafting priors (such as sparsity) for solving inverse problems is slowly being replaced by the use of richer learned priors (such as those modeled by generative adversarial networks, or …
LLM Sparsity Prior for Robust Feature Selection
Large language models (LLMs) offer a scalable mechanism to elicit domain-informed prior information for high-dimensional variable selection. However, existing methods such as LLM-Lasso are sensitive to weight quality, wi…
Prompt EngineeringProvably Convergent Algorithms for Solving Inverse Problems Using Generative Models
The traditional approach of hand-crafting priors (such as sparsity) for solving inverse problems is slowly being replaced by the use of richer learned priors (such as those modeled by deep generative networks). In this w…
A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors
Hierarchical models with gamma hyperpriors provide a flexible, sparse-promoting framework to bridge $L^1$ and $L^2$ regularizations in Bayesian formulations to inverse problems. Despite the Bayesian motivation for these …
Model SelectionTime SeriesTime Series AnalysisUncertainty Quantification+1