Variational Inference for Bayesian Bridge Regression
We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $\ell_{\alpha}$ norm, with $\alpha \in (0, +\infty)$ to define a penalization on large values of the regression coefficients, which includes the Lasso ($\alpha = 1$) and ridge $(\alpha = 2)$ penalizations as special cases. Full Bayesian inference seamlessly provides joint uncertainty estimates for all model parameters. Although MCMC aproaches are available for bridge regression, it can be slow for large dataset, specially in high dimensions. The ADVI implementation allows the use of small batches of data at each iteration (due to stochastic gradient based algorithms), therefore speeding up computational time in comparison with MCMC. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions. A simulation study shows the main properties of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceregressionVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational Bayesian inference for linear and logistic regression
The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of ac…
Bayesian InferenceregressionDiffusion Bridge Variational Inference for Deep Gaussian Processes
Deep Gaussian processes (DGPs) enable expressive hierarchical Bayesian modeling but pose substantial challenges for posterior inference, especially over inducing variables. Denoising diffusion variational inference (DDVI…
Image ReconstructionGaussian ProcessesDeep Variational Inference Symbolic Regression
Symbolic regression discovers explicit, interpretable equations without assuming a functional form in advance. A Bayesian approach strengthens this through probability distributions over candidate expressions, thus quant…
Neural Variational Gradient Descent
Particle-based approximate Bayesian inference approaches such as Stein Variational Gradient Descent (SVGD) combine the flexibility and convergence guarantees of sampling methods with the computational benefits of variati…
Bayesian InferenceregressionVariational InferenceGood Initializations of Variational Bayes for Deep Models
Stochastic variational inference is an established way to carry out approximate Bayesian inference for deep models. While there have been effective proposals for good initializations for loss minimization in deep learnin…
Bayesian InferenceGeneral ClassificationregressionVariational Inference