Conditional Deep Gaussian Processes: multi-fidelity kernel learning
Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model capable of a mathematically grounded estimation of uncertainty. The expressivity of DPGs results from not only the compositional character but the distribution propagation within the hierarchy. Recently, [1] pointed out that the hierarchical structure of DGP well suited modeling the multi-fidelity regression, in which one is provided sparse observations with high precision and plenty of low fidelity observations. We propose the conditional DGP model in which the latent GPs are directly supported by the fixed lower fidelity data. Then the moment matching method in [2] is applied to approximate the marginal prior of conditional DGP with a GP. The obtained effective kernels are implicit functions of the lower-fidelity data, manifesting the expressivity contributed by distribution propagation within the hierarchy. The hyperparameters are learned via optimizing the approximate marginal likelihood. Experiments with synthetic and high dimensional data show comparable performance against other multi-fidelity regression methods, variational inference, and multi-output GP. We conclude that, with the low fidelity data and the hierarchical DGP structure, the effective kernel encodes the inductive bias for true function allowing the compositional freedom discussed in [3,4].
Code (1)
Tasks
Few-Shot LearningGaussian ProcessesInductive BiasregressionTransfer LearningVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-fidelity Hierarchical Neural Processes
Science and engineering fields use computer simulation extensively. These simulations are often run at multiple levels of sophistication to balance accuracy and efficiency. Multi-fidelity surrogate modeling reduces the c…
EpidemiologyGaussian ProcessesBayesian Deconditional Kernel Mean Embeddings
Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the or…
Gaussian ProcessesKernel Conditional Density Operators
We introduce a novel conditional density estimation model termed the conditional density operator (CDO). It naturally captures multivariate, multimodal output densities and shows performance that is competitive with rece…
Density EstimationGaussian ProcessesMulti-fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces
Gaussian processes are employed for non-parametric regression in a Bayesian setting. They generalize linear regression, embedding the inputs in a latent manifold inside an infinite-dimensional reproducing kernel Hilbert …
Gaussian ProcessesregressionConditional Deep Gaussian Processes: empirical Bayes hyperdata learning
It is desirable to combine the expressive power of deep learning with Gaussian Process (GP) in one expressive Bayesian learning model. Deep kernel learning showed success in adopting a deep network for feature extraction…
Gaussian ProcessesVariational Inference