SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data
Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesTime SeriesTime Series AnalysisVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fantasizing with Dual GPs in Bayesian Optimization and Active Learning
Gaussian processes (GPs) are the main surrogate functions used for sequential modelling such as Bayesian Optimization and Active Learning. Their drawbacks are poor scaling with data and the need to run an optimization lo…
Active LearningBayesian OptimizationGaussian ProcessesSparse Gaussian Processes: Structured Approximations and Power-EP Revisited
Inducing-point-based sparse variational Gaussian processes have become the standard workhorse for scaling up GP models. Recent advances show that these methods can be improved by introducing a diagonal scaling matrix to …
Gaussian ProcessesRegularized Sparse Gaussian Processes
Gaussian processes are a flexible Bayesian nonparametric modelling approach that has been widely applied but poses computational challenges. To address the poor scaling of exact inference methods, approximation methods b…
Facial Expression Recognition (FER)Gaussian ProcessesImage ReconstructionPose Estimation+1Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation
Sparse Gaussian Processes are a key component of high-throughput Bayesian Optimisation (BO) loops; however, we show that existing methods for allocating their inducing points severely hamper optimisation performance. By …
Bayesian OptimisationDecision MakingDiversityGaussian Processes+2Prediction under Uncertainty in Sparse Spectrum Gaussian Processes with Applications to Filtering and Control
Sparse Spectrum Gaussian Processes (SSGPs) are a powerful tool for scaling Gaussian processes (GPs) to large datasets. Existing SSGP algorithms for regression assume deterministic inputs, precluding their use in man…
Gaussian ProcessesModel Predictive Controlregression