Fast Variational Block-Sparse Bayesian Learning
We present a variational Bayesian (VB) implementation of block-sparse Bayesian learning (BSBL), which approximates the posterior probability density function (PDF) of the latent variables a factorzied proxy PDFs. The prior distribution of the BSBL hyperparameters is selected to be the generalized inverse Gaussian distribution. This choice unifies commonly used hyperpriors, e.g. the Gamma distribution, inverse Gamma distribution, and Jeffrey's improper prior. The resulting variational BSBL (VA-BSBL) algorithm updates in an iterative manner the proxi PDFs of the weights and hyperparameters. The proxy PDF of the weights only depends on the proxy PDFs of the hyperparameters through the means of the latter PDFs. We adopt a scheduling of the iterations in which the proxi PDF of a single hyperparameter and the proxi PDF of the weights are cyclically updated ad infinitum. The resulting sequence of means of the hyperparameter proxi PDFs computed in this way can be expressed as a nonlinear first-order recurrence relation. Hence, the fixed points of the recurrence relation can be used for single-step check for convergence of this sequence. Including this check procedure in the VA-BSBL, we obtain a fast version of the algorithm, coined fast-BSBL (F-BSBL), which shows a two-order-of-magnitude faster runtime. Additionally, we analyse a necessary condition for the existence of fixed points and show that only certain improper generalized inverse Gaussian hyperpriors induce a sparse estimate of the weights. Finally, we generalize this equivalence to BSBL. Specifically, we show that the presented VA-BSBL is equivalent to expectation-maximization (EM)-based Type-II-BSBL. As a result, the fast versions of them, i.e. FBSBL for the former and Type-II-BSBL using coordinate ascent for the latter, are equivalent as well. This result allows for reinterpreting previous works on BSBL within our framework.
Code (1)
Tasks
SchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Variational EM Method for Pole-Zero Modeling of Speech with Mixed Block Sparse and Gaussian Excitation
The modeling of speech can be used for speech synthesis and speech recognition. We present a speech analysis method based on pole-zero modeling of speech with mixed block sparse and Gaussian excitation. By using a pole-z…
speech-recognitionSpeech RecognitionSpeech SynthesisFast Marginalized Block Sparse Bayesian Learning Algorithm
The performance of sparse signal recovery from noise corrupted, underdetermined measurements can be improved if both sparsity and correlation structure of signals are exploited. One typical correlation structure is the i…
Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature
In applied Bayesian inference scenarios, users may have access to a large number of pre-existing model evaluations, for example from maximum-a-posteriori (MAP) optimization runs. However, traditional approximate inferenc…
Active LearningBayesian InferenceVariational InferenceNonconvex Latent Optimally Partitioned Block-Sparse Recovery via Log-Sum and Minimax Concave Penalties
We propose two nonconvex regularization methods, LogLOP-l2/l1 and AdaLOP-l2/l1, for recovering block-sparse signals with unknown block partitions. These methods address the underestimation bias of existing convex approac…
Frequentist Consistency of Variational Bayes
A key challenge for modern Bayesian statistics is how to perform scalable inference of posterior distributions. To address this challenge, variational Bayes (VB) methods have emerged as a popular alternative to the class…