Doubly Stochastic Variational Inference for Deep Gaussian Processes
Gaussian processes (GPs) are a good choice for function approximation as they are flexible, robust to over-fitting, and provide well-calibrated predictive uncertainty. Deep Gaussian processes (DGPs) are multi-layer generalisations of GPs, but inference in these models has proved challenging. Existing approaches to inference in DGP models assume approximate posteriors that force independence between the layers, and do not work well in practice. We present a doubly stochastic variational inference algorithm, which does not force independence between layers. With our method of inference we demonstrate that a DGP model can be used effectively on data ranging in size from hundreds to a billion points. We provide strong empirical evidence that our inference scheme for DGPs works well in practice in both classification and regression.
Code (8)
Tasks
Gaussian ProcessesGeneral ClassificationregressionVariational InferenceSimilar Papers 제목 키워드 기반
Data Association with Gaussian Processes
The data association problem is concerned with separating data coming from different generating processes, for example when data come from different data sources, contain significant noise, or exhibit multimodality. We p…
Gaussian ProcessesVariational InferenceGeneralised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference
Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of …
BenchmarkingDimensionality ReductionGaussian ProcessesVariational InferencePartially Observable Gaussian Process Network and Doubly Stochastic Variational Inference
To reduce the curse of dimensionality for Gaussian processes (GP), they can be decomposed into a Gaussian Process Network (GPN) of coupled subprocesses with lower dimensionality. In some cases, intermediate observations …
Gaussian ProcessesVariational InferenceDeep Gaussian Processes for Air Quality Inference
Air pollution kills around 7 million people annually, and approximately 2.4 billion people are exposed to hazardous air pollution. Accurate, fine-grained air quality (AQ) monitoring is essential to control and reduce pol…
Air Quality InferenceGaussian ProcessesVariational InferenceDoubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables
Neural processes (NPs) constitute a family of variational approximate models for stochastic processes with promising properties in computational efficiency and uncertainty quantification. These processes use neural netwo…
Computational EfficiencyregressionUncertainty QuantificationVariational Inference