High-Dimensional Sparse Bayesian Learning without Covariance Matrices
Sparse Bayesian learning (SBL) is a powerful framework for tackling the sparse coding problem. However, the most popular inference algorithms for SBL become too expensive for high-dimensional settings, due to the need to store and compute a large covariance matrix. We introduce a new inference scheme that avoids explicit construction of the covariance matrix by solving multiple linear systems in parallel to obtain the posterior moments for SBL. Our approach couples a little-known diagonal estimation result from numerical linear algebra with the conjugate gradient algorithm. On several simulations, our method scales better than existing approaches in computation time and memory, especially for structured dictionaries capable of fast matrix-vector multiplication.
Code (1)
Tasks
Vocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Covariance-Free Sparse Bayesian Learning
Sparse Bayesian learning (SBL) is a powerful framework for tackling the sparse coding problem while also providing uncertainty quantification. The most popular inference algorithms for SBL exhibit prohibitively large com…
compressed sensingMRI ReconstructionUncertainty QuantificationA Bayesian sparse factor model with adaptive posterior concentration
In this paper, we propose a new Bayesian inference method for a high-dimensional sparse factor model that allows both the factor dimensionality and the sparse structure of the loading matrix to be inferred. The novelty i…
Bayesian InferenceComplexity-Optimized Sparse Bayesian Learning for Scalable Classification Tasks
Sparse Bayesian Learning (SBL) constructs an extremely sparse probabilistic model with very competitive generalization. However, SBL needs to invert a big covariance matrix with complexity $O(M^3)$ (M: feature size) for …
Classificationfeature selectionBayesian Analysis for Over-parameterized Linear Model via Effective Spectra
In high-dimensional Bayesian statistics, various methods have been developed, including prior distributions that induce parameter sparsity to handle many parameters. Yet, these approaches often overlook the rich spectral…
Uncertainty QuantificationOn the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA
We study high-dimensional differentially private (DP) covariance estimation in the operator norm, and principal component analysis (PCA), under $k$-row-column sparsity ($k$-RCS) of the covariance matrix. In the non-priva…