Variable Selection with Rigorous Uncertainty Quantification using Deep Bayesian Neural Networks: Posterior Concentration and Bernstein-von Mises Phenomenon
This work develops rigorous theoretical basis for the fact that deep Bayesian neural network (BNN) is an effective tool for high-dimensional variable selection with rigorous uncertainty quantification. We develop new Bayesian non-parametric theorems to show that a properly configured deep BNN (1) learns the variable importance effectively in high dimensions, and its learning rate can sometimes "break" the curse of dimensionality. (2) BNN's uncertainty quantification for variable importance is rigorous, in the sense that its 95% credible intervals for variable importance indeed covers the truth 95% of the time (i.e., the Bernstein-von Mises (BvM) phenomenon). The theoretical results suggest a simple variable selection algorithm based on the BNN's credible intervals. Extensive simulation confirms the theoretical findings and shows that the proposed algorithm outperforms existing classic and neural-network-based variable selection methods, particularly in high dimensions.
Code (0)
등록된 구현이 없습니다.
Tasks
Uncertainty QuantificationVariable SelectionSimilar Papers 제목 키워드 기반
Variational Bayes for high-dimensional proportional hazards models with applications within gene expression
Few Bayesian methods for analyzing high-dimensional sparse survival data provide scalable variable selection, effect estimation and uncertainty quantification. Such methods often either sacrifice uncertainty quantificati…
Uncertainty QuantificationVariable SelectionConformalized Generative Bayesian Imaging: An Uncertainty Quantification Framework for Computational Imaging
Uncertainty quantification plays an important role in achieving trustworthy and reliable learning-based computational imaging. Recent advances in generative modeling and Bayesian neural networks have enabled the developm…
Conformal PredictionImage InpaintingImage ReconstructionUncertainty QuantificationConvergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery
Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limi…
Model DiscoveryregressionUncertainty Quantificationvalid+1Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees
We develop a simple and unified framework for nonlinear variable selection that incorporates uncertainty in the prediction function and is compatible with a wide range of machine learning models (e.g., tree ensembles, ke…
Variable SelectionPartial Trace-Class Bayesian Neural Networks
Bayesian neural networks (BNNs) allow rigorous uncertainty quantification in deep learning, but often come at a prohibitive computational cost. We propose three different innovative architectures of partial trace-class B…