Deep kernel processes
We define deep kernel processes in which positive definite Gram matrices are progressively transformed by nonlinear kernel functions and by sampling from (inverse) Wishart distributions. Remarkably, we find that deep Gaussian processes (DGPs), Bayesian neural networks (BNNs), infinite BNNs, and infinite BNNs with bottlenecks can all be written as deep kernel processes. For DGPs the equivalence arises because the Gram matrix formed by the inner product of features is Wishart distributed, and as we show, standard isotropic kernels can be written entirely in terms of this Gram matrix -- we do not need knowledge of the underlying features. We define a tractable deep kernel process, the deep inverse Wishart process, and give a doubly-stochastic inducing-point variational inference scheme that operates on the Gram matrices, not on the features, as in DGPs. We show that the deep inverse Wishart process gives superior performance to DGPs and infinite BNNs on standard fully-connected baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesVariational InferenceSimilar Papers 제목 키워드 기반
Decoupled Kernel Neural Processes: Neural Network-Parameterized Stochastic Processes using Explicit Data-driven Kernel
Neural Processes (NPs) are a class of stochastic processes parametrized by neural networks. Unlike traditional stochastic processes (e.g., Gaussian processes), which require specifying explicit kernel functions, NPs impl…
DiversityGaussian ProcessesOn power sum kernels on symmetric groups
In this note, we introduce a family of "power sum" kernels and the corresponding Gaussian processes on symmetric groups $\mathrm{S}_n$. Such processes are bi-invariant: the action of $\mathrm{S}_n$ on itself from both si…
Gaussian ProcessesKriging via variably scaled kernels
Classical Gaussian processes and Kriging models are commonly based on stationary kernels, whereby correlations between observations depend exclusively on the relative distance between scattered data. While this assumptio…
Gaussian ProcessesSentiment analysis with genetically evolved Gaussian kernels
Sentiment analysis consists of evaluating opinions or statements from the analysis of text. Among the methods used to estimate the degree in which a text expresses a given sentiment, are those based on Gaussian Processes…
Gaussian ProcessesSentiment AnalysisMake Hawkes Processes Explainable by Decomposing Self-Triggering Kernels
Hawkes Processes capture self-excitation and mutual-excitation between events when the arrival of an event makes future events more likely to happen. Identification of such temporal covariance can reveal the underlying s…