Deep Gaussian Processes for Regression using Approximate Expectation Propagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers. DGPs are nonparametric probabilistic models and as such are arguably more flexible, have a greater capacity to generalise, and provide better calibrated uncertainty estimates than alternative deep models. This paper develops a new approximate Bayesian learning scheme that enables DGPs to be applied to a range of medium to large scale regression problems for the first time. The new method uses an approximate Expectation Propagation procedure and a novel and efficient extension of the probabilistic backpropagation algorithm for learning. We evaluate the new method for non-linear regression on eleven real-world datasets, showing that it always outperforms GP regression and is almost always better than state-of-the-art deterministic and sampling-based approximate inference methods for Bayesian neural networks. As a by-product, this work provides a comprehensive analysis of six approximate Bayesian methods for training neural networks.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesregressionSimilar Papers 제목 키워드 기반
Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers. DGPs are probabilistic …
Gaussian ProcessesQuantile Propagation for Wasserstein-Approximate Gaussian Processes
Approximate inference techniques are the cornerstone of probabilistic methods based on Gaussian process priors. Despite this, most work approximately optimizes standard divergence measures such as the Kullback-Leibler (K…
Bayesian InferenceGaussian ProcessesApproximate Inference for Nonstationary Heteroscedastic Gaussian process Regression
This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression. Our efficient and straightforward approach can also be applied to i…
regressionA Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
Gaussian processes (GPs) are flexible distributions over functions that enable high-level assumptions about unknown functions to be encoded in a parsimonious, flexible and general way. Although elegant, the application o…
Gaussian ProcessesSparse Gaussian Processes: Structured Approximations and Power-EP Revisited
Inducing-point-based sparse variational Gaussian processes have become the standard workhorse for scaling up GP models. Recent advances show that these methods can be improved by introducing a diagonal scaling matrix to …
Gaussian Processes