Variational Inference via Resolution of Singularities
Predicated on the premise that neural networks are best viewed as singular statistical models, we set out to propose a new variational approximation for Bayesian neural networks. The approximation relies on a central result from singular learning theory according to which the posterior distribution over the parameters of a singular model, following an algebraic-geometrical transformation known as a desingularization map, is asymptotically a mixture of standard forms. From here we proceed to demonstrate that a generalized gamma mean-field variational family, following desingularization, can recover the leading order term of the model evidence. Affine coupling layers are employed to learn the unknown desingularization map, effectively rendering the proposed methodology a normalizing flow with the generalized gamma as the source distribution.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning TheoryVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational Bayesian Neural Networks via Resolution of Singularities
In this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs). To begin, using SLT, we lay to rest so…
Learning TheoryVariational InferenceAn ML approach to resolution of singularities
The solution set of a system of polynomial equations typically contains ill-behaved, singular points. Resolution is a fundamental process in geometry in which we replace singular points with smooth points, while keeping …
An Edge Driven Wavelet Frame Model for Image Restoration
Wavelet frame systems are known to be effective in capturing singularities from noisy and degraded images. In this paper, we introduce a new edge driven wavelet frame model for image restoration by approximating images a…
DeblurringImage InpaintingImage RestorationExponential ReLU Neural Network Approximation Rates for Point and Edge Singularities
We prove exponential expressivity with stable ReLU Neural Networks (ReLU NNs) in $H^1(\Omega)$ for weighted analytic function classes in certain polytopal domains $\Omega$, in space dimension $d=2,3$. Functions in these …
Topological Singularity Detection at Multiple Scales
The manifold hypothesis, which assumes that data lies on or close to an unknown manifold of low intrinsic dimension, is a staple of modern machine learning research. However, recent work has shown that real-world data ex…