Compositional uncertainty in deep Gaussian processes
Gaussian processes (GPs) are nonparametric priors over functions. Fitting a GP implies computing a posterior distribution of functions consistent with the observed data. Similarly, deep Gaussian processes (DGPs) should allow us to compute a posterior distribution of compositions of multiple functions giving rise to the observations. However, exact Bayesian inference is intractable for DGPs, motivating the use of various approximations. We show that the application of simplifying mean-field assumptions across the hierarchy leads to the layers of a DGP collapsing to near-deterministic transformations. We argue that such an inference scheme is suboptimal, not taking advantage of the potential of the model to discover the compositional structure in the data. To address this issue, we examine alternative variational inference schemes allowing for dependencies across different layers and discuss their advantages and limitations.
Code (1)
Tasks
Bayesian InferenceGaussian ProcessesVariational InferenceSimilar Papers 제목 키워드 기반
Deep Sigma Point Processes
We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs). Deep Sigma Point Processes (DSPPs) retain many of the attractive features o…
Gaussian ProcessesPoint ProcessesregressionLearning Switching Port-Hamiltonian Systems with Uncertainty Quantification
Switching physical systems are ubiquitous in modern control applications, for instance, locomotion behavior of robots and animals, power converters with switches and diodes. The dynamics and switching conditions are ofte…
Gaussian ProcessesUncertainty QuantificationConditional Deep Gaussian Processes: multi-fidelity kernel learning
Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model capable of a mathematically grounded estimation of uncertainty. The expressivity of DPGs results from not only the compositional character but …
Few-Shot LearningGaussian ProcessesInductive Biasregression+2Deep Gaussian Processes for Multi-fidelity Modeling
Multi-fidelity methods are prominently used when cheaply-obtained, but possibly biased and noisy, observations must be effectively combined with limited or expensive true data in order to construct reliable models. This …
Bayesian OptimizationDecision MakingGaussian ProcessesVariational InferenceDeep Compositional Spatial Models
Spatial processes with nonstationary and anisotropic covariance structure are often used when modelling, analysing and predicting complex environmental phenomena. Such processes may often be expressed as ones that have s…
Gaussian ProcessesUncertainty Quantification