Attentive Gaussian processes for probabilistic time-series generation
The transduction of sequence has been mostly done by recurrent networks, which are computationally demanding and often underestimate uncertainty severely. We propose a computationally efficient attention-based network combined with the Gaussian process regression to generate real-valued sequence, which we call the Attentive-GP. The proposed model not only improves the training efficiency by dispensing recurrence and convolutions but also learns the factorized generative distribution with Bayesian representation. However, the presence of the GP precludes the commonly used mini-batch approach to the training of the attention network. Therefore, we develop a block-wise training algorithm to allow mini-batch training of the network while the GP is trained using full-batch, resulting in a scalable training method. The algorithm has been proved to converge and shows comparable, if not better, quality of the found solution. As the algorithm does not assume any specific network architecture, it can be used with a wide range of hybrid models such as neural networks with kernel machine layers in the scarcity of resources for computation and memory.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesTime SeriesTime Series AnalysisTime Series GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Probabilistic Data-Driven Modelling of Astrophysical Transients: The Neural Process Family for Ultrafast and Class-Agnostic Light Curve Reconstruction with NightLANP
Astrophysical observations from Earth are subject to weather, environmental, and scientific constraints that lead to sparse, irregular light curves. On the eve of the Vera C. Rubin Observatory Legacy Survey of Space and …
Gaussian ProcessesLearning to Forget: Bayesian Time Series Forecasting using Recurrent Sparse Spectrum Signature Gaussian Processes
The signature kernel is a kernel between time series of arbitrary length and comes with strong theoretical guarantees from stochastic analysis. It has found applications in machine learning such as covariance functions f…
Gaussian ProcessesGPUProbabilistic Time Series ForecastingTime Series+2Forecasting intermittent time series with Gaussian Processes and Tweedie likelihood
We adopt Gaussian Processes (GPs) as latent functions for probabilistic forecasting of intermittent time series. The model is trained in a Bayesian framework that accounts for the uncertainty about the latent function an…
Gaussian ProcessesTime SeriesFlow Matching with Gaussian Process Priors for Probabilistic Time Series Forecasting
Recent advancements in generative modeling, particularly diffusion models, have opened new directions for time series modeling, achieving state-of-the-art performance in forecasting and synthesis. However, the reliance o…
Gaussian ProcessesProbabilistic Time Series ForecastingTime SeriesTime Series ForecastingTaylorformer: Probabilistic Modelling for Random Processes including Time Series
We propose the Taylorformer for random processes such as time series. Its two key components are: 1) the LocalTaylor wrapper which adapts Taylor approximations (used in dynamical systems) for use in neural network-based …
Gaussian ProcessesMeta-LearningTime Series