Incremental Variational Sparse Gaussian Process Regression
Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference. However, the majority of these approaches are batch methods that operate on the entire training dataset at once, precluding the use of datasets that are streaming or too large to fit into memory. Although previous work has considered incrementally solving variational sparse GPR, most algorithms fail to update the basis functions and therefore perform suboptimally. We propose a novel incremental learning algorithm for variational sparse GPR based on stochastic mirror ascent of probability densities in reproducing kernel Hilbert space. This new formulation allows our algorithm to update basis functions online in accordance with the manifold structure of probability densities for fast convergence. We conduct several experiments and show that our proposed approach achieves better empirical performance in terms of prediction error than the recent state-of-the-art incremental solutions to variational sparse GPR.
Code (0)
등록된 구현이 없습니다.
Tasks
GPRIncremental LearningregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Decoupled Sparse Gaussian Processes Components]{Decoupled Sparse Gaussian Processes Components : Separating Decision Making from Data Manifold Fitting
We propose a decoupling in Reproducing Kernel Hilbert Space of the parametric and non-parametric components of Sparse Gaussian Processes. We demonstrate that this decoupling results in a set of inducing points concentra…
Decision MakingGaussian ProcessesregressionWasserstein-Splitting Gaussian Process Regression for Heterogeneous Online Bayesian Inference
Gaussian processes (GPs) are a well-known nonparametric Bayesian inference technique, but they suffer from scalability problems for large sample sizes, and their performance can degrade for non-stationary or spatially he…
Bayesian InferenceGaussian ProcessesregressionSparse Variational Contaminated Noise Gaussian Process Regression with Applications in Geomagnetic Perturbations Forecasting
Gaussian Processes (GP) have become popular machine-learning methods for kernel-based learning on datasets with complicated covariance structures. In this paper, we present a novel extension to the GP framework using a c…
Gaussian ProcessesPrediction IntervalsregressionIncremental Nonlinear System Identification and Adaptive Particle Filtering Using Gaussian Process
An incremental/online state dynamic learning method is proposed for identification of the nonlinear Gaussian state space models. The method embeds the stochastic variational sparse Gaussian process as the probabilistic s…
State EstimationState Space ModelsSparse Orthogonal Variational Inference for Gaussian Processes
We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points, which can lead to more scalable algorithms than previous methods. It is based on decomposing a Gaussian…
Gaussian ProcessesMulti-class ClassificationregressionVariational Inference