Neural Linear Models with Functional Gaussian Process Priors
Neural linear models (NLM) and Gaussian processes (GP) are both examples of Bayesian linear regression on rich feature spaces. In contrast to the widespread use of nonparametric GPs for probabilistic nonlinear regression, NLMs remain an underused parametric alternative because standard type II maximum likelihood (ML) training leads to overconfidence outside of the data distribution. Therefore, we propose to augment this training procedure through functional variational inference (fVI) proposed by Sun et. al. (2019), which is particularly well suited for NLMs due to their closed-form predictive distribution. Additionally, we investigate whether an appropriate functional prior can guide parametric NLMs to attain nonparametric GP performance, despite using fewer parameters. Results show that functional priors do improve performance of NLM over ML training, and that the NLM performs on par with weight space BNNs in this setting.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesregressionVariational InferenceSimilar Papers 제목 키워드 기반
Functional Priors for Bayesian Neural Networks through Wasserstein Distance Minimization to Gaussian Processes
The Bayesian treatment of neural networks dictates that a prior distribution is considered over the weight and bias parameters of the network. The non-linear nature of the model implies that any distribution of the param…
Gaussian ProcessesHybrid Bayesian Neural Networks with Functional Probabilistic Layers
Bayesian neural networks provide a direct and natural way to extend standard deep neural networks to support probabilistic deep learning through the use of probabilistic layers that, traditionally, encode weight (and bia…
Bayesian InferenceGaussian ProcessesProbabilistic Deep LearningVariational InferenceUniversal Functional Regression with Neural Operator Flows
Regression on function spaces is typically limited to models with Gaussian process priors. We introduce the notion of universal functional regression, in which we aim to learn a prior distribution over non-Gaussian funct…
Gaussian ProcessesregressionUncertainty QuantificationOptimal experimental design via Bayesian optimization: active causal structure learning for Gaussian process networks
We study the problem of causal discovery through targeted interventions. Starting from few observational measurements, we follow a Bayesian active learning approach to perform those experiments which, in expectation with…
Active LearningBayesian OptimisationBayesian OptimizationCausal Discovery+1Varying-coefficient models with isotropic Gaussian process priors
We study learning problems in which the conditional distribution of the output given the input varies as a function of additional task variables. In varying-coefficient models with Gaussian process priors, a Gaussian pro…
Bayesian Inference