Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation

2017-11-02 · Roman Föll, Bernard Haasdonk, Markus Hanselmann, Holger Ulmer

Modeling sequential data has become more and more important in practice. Some applications are autonomous driving, virtual sensors and weather forecasting. To model such systems so called recurrent models are used. In this article we introduce two new Deep Recurrent Gaussian Process (DRGP) models based on the Sparse Spectrum Gaussian Process (SSGP) and the improved variational version called Variational Sparse Spectrum Gaussian Process (VSSGP). We follow the recurrent structure given by an existing DRGP based on a specific sparse Nystr\"om approximation. Therefore, we also variationally integrate out the input-space and hence can propagate uncertainty through the layers. We can show that for the resulting lower bound an optimal variational distribution exists. Training is realized through optimizing the variational lower bound. Using Distributed Variational Inference (DVI), we can reduce the computational complexity. We improve over current state of the art methods in prediction accuracy for experimental data-sets used for their evaluation and introduce a new data-set for engine control, named Emission. Furthermore, our method can easily be adapted for unsupervised learning, e.g. the latent variable model and its deep version.

📄 PDF Abstract BibTeX arXiv:1711.00799

Code (1)

RomanFoell/DRGP-VSS 공식 구현

Tasks

Autonomous DrivingVariational InferenceWeather Forecasting

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Deep recurrent Gaussian process with variational Sparse Spectrum approximation

2019-09-27 · Roman Föll, Bernard Haasdonk, Markus Hanselmann, Holger Ulmer

Autonomous Driving Weather Forecasting

Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs

2015-03-09 · Yarin Gal, Richard Turner

Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variatio…

Variational Inference

A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression

2016-11-18 · Quang Minh Hoang, Trong Nghia Hoang, Kian Hsiang Low

While much research effort has been dedicated to scaling up sparse Gaussian process (GP) models based on inducing variables for big data, little attention is afforded to the other less explored class of low-rank GP appro…

regression Stochastic Optimization

Learning to Forget: Bayesian Time Series Forecasting using Recurrent Sparse Spectrum Signature Gaussian Processes

2024-12-27 · Csaba Tóth, Masaki Adachi, Michael A. Osborne, Harald Oberhauser

The signature kernel is a kernel between time series of arbitrary length and comes with strong theoretical guarantees from stochastic analysis. It has found applications in machine learning such as covariance functions f…

Gaussian Processes GPU Probabilistic Time Series Forecasting Time Series+2

Modelling Non-Smooth Signals with Complex Spectral Structure

2022-03-14 · Wessel P. Bruinsma, Martin Tegnér, Richard E. Turner

The Gaussian Process Convolution Model (GPCM; Tobar et al., 2015a) is a model for signals with complex spectral structure. A significant limitation of the GPCM is that it assumes a rapidly decaying spectrum: it can only …