A Study of Posterior Stability for Time-Series Latent Diffusion
Latent diffusion has demonstrated promising results in image generation and permits efficient sampling. However, this framework might suffer from the problem of posterior collapse when applied to time series. In this paper, we first show that posterior collapse will reduce latent diffusion to a variational autoencoder (VAE), making it less expressive. This highlights the importance of addressing this issue. We then introduce a principled method: dependency measure, that quantifies the sensitivity of a recurrent decoder to input variables. Using this tool, we confirm that posterior collapse significantly affects time-series latent diffusion on real datasets, and a phenomenon termed dependency illusion is also discovered in the case of shuffled time series. Finally, building on our theoretical and empirical studies, we introduce a new framework that extends latent diffusion and has a stable posterior. Extensive experiments on multiple real time-series datasets show that our new framework is free from posterior collapse and significantly outperforms previous baselines in time series synthesis.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderImage GenerationTime SeriesTime Series GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Black box variational inference for state space models
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from th…
State Space ModelsTime SeriesTime Series AnalysisVariational InferenceVariational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
We propose a recurrent neural network for a "model-free" simulation of a dynamical system with unknown parameters without prior knowledge. The deep learning model aims to jointly learn the nonlinear time marching operato…
Time SeriesTime Series AnalysisVariational InferenceExploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework
The Probabilistic Transformer (PT) establishes that the Transformer's self-attention plus its feed-forward block is mathematically equivalent to Mean-Field Variational Inference (MFVI) on a Conditional Random Field (CRF)…
Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection
Multivariate time series anomaly detection has become an active area of research in recent years, with Deep Learning models outperforming previous approaches on benchmark datasets. Among reconstruction-based models, most…
Anomaly DetectionMissing ValuesTime SeriesTime Series Analysis+1Improving Variational Auto-Encoders using Householder Flow
Variational auto-encoders (VAE) are scalable and powerful generative models. However, the choice of the variational posterior determines tractability and flexibility of the VAE. Commonly, latent variables are modeled usi…
Computational Efficiency