Sparse Gaussian processes using pseudo-inputs
We present a new Gaussian process (GP) regression model whose covariance is parameterized by the the locations of M pseudo-input points, which we learn by a gradient based optimization. We take M << N, where N is the number of real data points, and hence obtain a sparse regression method which has O(M^2N) training cost and O(M^2) prediction cost per test case. We also find hyperparameters of the covariance function in the same joint optimization. The method can be viewed as a Bayesian regression model with particular input dependent noise. The method turns out to be closely related to several other sparse GP approaches, and we discuss the relation in detail. We finally demonstrate its performance on some large data sets, and make a direct comparison to other sparse GP methods. We show that our method can match full GP performance with small M, i.e. very sparse solutions, and it significantly outperforms other approaches in this regime.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesregressionSimilar Papers 제목 키워드 기반
Deep Gaussian Processes with Decoupled Inducing Inputs
Deep Gaussian Processes (DGP) are hierarchical generalizations of Gaussian Processes (GP) that have proven to work effectively on a multiple supervised regression tasks. They combine the well calibrated uncertainty estim…
Gaussian ProcessesVariable noise and dimensionality reduction for sparse Gaussian processes
The sparse pseudo-input Gaussian process (SPGP) is a new approximation method for speeding up GP regression in the case of a large number of data points N. The approximation is controlled by the gradient optimization of …
Dimensionality ReductionGaussian ProcessesRecyclable Gaussian Processes
We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes wi…
Gaussian ProcessesregressionModular Gaussian Processes for Transfer Learning
We present a framework for transfer learning based on modular variational Gaussian processes (GP). We develop a module-based method that having a dictionary of well fitted GPs, one could build ensemble GP models without …
Gaussian ProcessesTransfer LearningvalidFunctional Regularisation for Continual Learning with Gaussian Processes
We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Cont…
Bayesian InferenceContinual LearningGaussian ProcessesSequential Bayesian Inference