Uncertainty Quantification for Sparse Deep Learning
Deep learning methods continue to have a decided impact on machine learning, both in theory and in practice. Statistical theoretical developments have been mostly concerned with approximability or rates of estimation when recovering infinite dimensional objects (curves or densities). Despite the impressive array of available theoretical results, the literature has been largely silent about uncertainty quantification for deep learning. This paper takes a step forward in this important direction by taking a Bayesian point of view. We study Gaussian approximability of certain aspects of posterior distributions of sparse deep ReLU architectures in non-parametric regression. Building on tools from Bayesian non-parametrics, we provide semi-parametric Bernstein-von Mises theorems for linear and quadratic functionals, which guarantee that implied Bayesian credible regions have valid frequentist coverage. Our results provide new theoretical justifications for (Bayesian) deep learning with ReLU activation functions, highlighting their inferential potential.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningUncertainty QuantificationvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery
Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limi…
Model DiscoveryregressionUncertainty Quantificationvalid+1Uncertainty Quantification for Scientific Machine Learning using Sparse Variational Gaussian Process Kolmogorov-Arnold Networks (SVGP KAN)
Kolmogorov-Arnold Networks have emerged as interpretable alternatives to traditional multi-layer perceptrons. However, standard implementations lack principled uncertainty quantification capabilities essential for many s…
Out-of-Distribution DetectionBayesian InferenceSparse Deep Learning for Time Series Data: Theory and Applications
Sparse deep learning has become a popular technique for improving the performance of deep neural networks in areas such as uncertainty quantification, variable selection, and large-scale network compression. However, mos…
Deep LearningModel CompressionTime SeriesUncertainty Quantification+1UQ-SHRED: uncertainty quantification of shallow recurrent decoder networks for sparse sensing via engression
Reconstructing high-dimensional spatiotemporal fields from sparse sensor measurements is critical in a wide range of scientific applications. The SHallow REcurrent Decoder (SHRED) architecture is a recent state-of-the-ar…
Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion prior
We study pointwise estimation and uncertainty quantification for a sparse variational Gaussian process method with eigenvector inducing variables. For a rescaled Brownian motion prior, we derive theoretical guarantees an…
Uncertainty Quantification