Functional Priors for Bayesian Neural Networks through Wasserstein Distance Minimization to Gaussian Processes
The Bayesian treatment of neural networks dictates that a prior distribution is considered over the weight and bias parameters of the network. The non-linear nature of the model implies that any distribution of the parameters has an unpredictable effect on the distribution of the function output. Gaussian processes offer a rigorous framework to define prior distributions over the space of functions. Our proposal is to impose such functional priors on well-established architectures of neural networks by means of minimising the Wasserstein distance between samples of stochastic processes. Early experimental results demonstrate the potential of functional priors for Bayesian neural networks.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesSimilar Papers 제목 키워드 기반
All You Need is a Good Functional Prior for Bayesian Deep Learning
The Bayesian treatment of neural networks dictates that a prior distribution is specified over their weight and bias parameters. This poses a challenge because modern neural networks are characterized by a large number o…
AllGaussian ProcessesLikelihood-Free Adaptive Bayesian Inference via Nonparametric Distribution Matching
When the likelihood is analytically unavailable and computationally intractable, approximate Bayesian computation (ABC) has emerged as a widely used methodology for approximate posterior inference; however, it suffers fr…
Bayesian InferenceDensity Estimationquantile regressionConditional Wasserstein Distances with Applications in Bayesian OT Flow Matching
In inverse problems, many conditional generative models approximate the posterior measure by minimizing a distance between the joint measure and its learned approximation. While this approach also controls the distance b…
Conditional Image GenerationImage GenerationPAC-Bayesian Generalization Bounds for Adversarial Generative Models
We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the…
Dimensionality ReductionGeneralization BoundsShedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances
The Sliced-Wasserstein distance (SW) is a computationally efficient and theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its statistical properties -- or, more accurately, its genera…
Generalization Bounds