Analytic solution and stationary phase approximation for the Bayesian lasso and elastic net
The lasso and elastic net linear regression models impose a double-exponential prior distribution on the model parameters to achieve regression shrinkage and variable selection, allowing the inference of robust models from large data sets. However, there has been limited success in deriving estimates for the full posterior distribution of regression coefficients in these models, due to a need to evaluate analytically intractable partition function integrals. Here, the Fourier transform is used to express these integrals as complex-valued oscillatory integrals over "regression frequencies". This results in an analytic expansion and stationary phase approximation for the partition functions of the Bayesian lasso and elastic net, where the non-differentiability of the double-exponential prior has so far eluded such an approach. Use of this approximation leads to highly accurate numerical estimates for the expectation values and marginal posterior distributions of the regression coefficients, and allows for Bayesian inference of much higher dimensional models than previously possible.
Code (1)
Tasks
Bayesian InferenceregressionVariable SelectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Continual Learning with Bayesian Neural Networks for Non-Stationary Data
This work addresses continual learning for non-stationary data, using Bayesian neural networks and memory-based online variational Bayes. We represent the posterior approximation of the network weights by a diagonal Gaus…
Continual LearningOn the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers
We study stationary solutions of McKean-Vlasov equations on the circle. Our main contributions stem from observing an exact equivalence between solutions of the stationary McKean-Vlasov equation and an infinite-dimension…
Analytical Approximation of the ELBO Gradient in the Context of the Clutter Problem
We propose an analytical solution for approximating the gradient of the Evidence Lower Bound (ELBO) in variational inference problems where the statistical model is a Bayesian network consisting of observations drawn fro…
Bayesian InferenceVariational InferenceAnalytics of Business Time Series Using Machine Learning and Bayesian Inference
In the survey we consider the case studies on sales time series forecasting, the deep learning approach for forecasting non-stationary time series using time trend correction, dynamic price and supply optimization using …
Bayesian InferenceBIG-bench Machine LearningQ-LearningSurvey+3Discovering Inductive Bias with Gibbs Priors: A Diagnostic Tool for Approximate Bayesian Inference
Full Bayesian posteriors are rarely analytically tractable, which is why real-world Bayesian inference heavily relies on approximate techniques. Approximations generally differ from the true posterior and require diagnos…
Bayesian InferenceDiagnosticInductive Bias