Horseshoe Regularization for Machine Learning in Complex and Deep Models
Since the advent of the horseshoe priors for regularization, global-local shrinkage methods have proved to be a fertile ground for the development of Bayesian methodology in machine learning, specifically for high-dimensional regression and classification problems. They have achieved remarkable success in computation, and enjoy strong theoretical support. Most of the existing literature has focused on the linear Gaussian case; see Bhadra et al. (2019b) for a systematic survey. The purpose of the current article is to demonstrate that the horseshoe regularization is useful far more broadly, by reviewing both methodological and computational developments in complex models that are more relevant to machine learning applications. Specifically, we focus on methodological challenges in horseshoe regularization in nonlinear and non-Gaussian models; multivariate models; and deep neural networks. We also outline the recent computational developments in horseshoe shrinkage for complex models along with a list of available software implementations that allows one to venture out beyond the comfort zone of the canonical linear regression problems.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
False Discovery Rate Control via Frequentist-assisted Horseshoe
The horseshoe prior, a widely used handy alternative to the spike-and-slab prior, has proven to be an exceptional default global-local shrinkage prior in Bayesian inference and machine learning. However, designing tests …
Bayesian InferenceBayesian Regularization for Functional Graphical Models with Applications to Neuroimaging
Graphical models, used to express conditional dependence between random variables observed at various nodes, are used extensively in many fields such as genetics, neuroscience, and social network analysis. While most cur…
Tree Ensembles with Rule Structured Horseshoe Regularization
We propose a new Bayesian model for flexible nonlinear regression and classification using tree ensembles. The model is based on the RuleFit approach in Friedman and Popescu (2008) where rules from decision trees and lin…
Cancer ClassificationGeneral ClassificationregressionHorseshoe Regularization for Feature Subset Selection
Feature subset selection arises in many high-dimensional applications of statistics, such as compressed sensing and genomics. The $\ell_0$ penalty is ideal for this task, the caveat being it requires the NP-hard combinat…
compressed sensingUncertainty QuantificationT-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs
Graphs have been commonly used to represent complex data structures. In models dealing with graph-structured data, multivariate parameters may not only exhibit sparse patterns but have structured sparsity and smoothness …
Anomaly DetectionBayesian Inferenceregression