Sparse Methods for Automatic Relevance Determination
This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal covariates, we analytically demonstrate favorable performance with regards to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.
Code (1)
Tasks
regressionSimilar Papers 제목 키워드 기반
Sparse Bayesian structure learning with “dependent relevance determination” priors
In many problem settings, parameter vectors are not merely sparse, but dependent in such a way that non-zero coefficients tend to cluster together. We refer to this form of dependency as “region sparsity”. Classical spar…
regressionCorrentropy-Based Logistic Regression with Automatic Relevance Determination for Robust Sparse Brain Activity Decoding
Recent studies have utilized sparse classifications to predict categorical variables from high-dimensional brain activity signals to expose human's intentions and mental states, selecting the relevant features automatica…
Brain DecodingClassificationEEGElectroencephalogram (EEG)+1Dependent relevance determination for smooth and structured sparse regression
In many problem settings, parameter vectors are not merely sparse but dependent in such a way that non-zero coefficients tend to cluster together. We refer to this form of dependency as "region sparsity." Classical spars…
regressionAdaptive sparseness for correntropy-based robust regression via automatic relevance determination
Sparseness and robustness are two important properties for many machine learning scenarios. In the present study, regarding the maximum correntropy criterion (MCC) based robust regression algorithm, we investigate to int…
Bayesian Inferencefeature selectionregressionLaplacian Prior Variational Automatic Relevance Determination for Transmission Tomography
In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice…