Learning Stochastic Recurrent Networks
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a reliable estimator of the marginal likelihood and iv) is a generalisation of deterministic recurrent neural networks. We evaluate the method on four polyphonic musical data sets and motion capture data.
Code (1)
Tasks
Variational InferenceSimilar Papers 제목 키워드 기반
Stochastic Recurrent Neural Network for Multistep Time Series Forecasting
Time series forecasting based on deep architectures has been gaining popularity in recent years due to their ability to model complex non-linear temporal dynamics. The recurrent neural network is one such model capable o…
State Space ModelsTime SeriesTime Series AnalysisTime Series ForecastingRe-examination of the Role of Latent Variables in Sequence Modeling
With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networ…
Density EstimationSequential Neural Models with Stochastic Layers
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network…
Music ModelingVariational InferenceTowards Recurrent Autoregressive Flow Models
Stochastic processes generated by non-stationary distributions are difficult to represent with conventional models such as Gaussian processes. This work presents Recurrent Autoregressive Flows as a method toward general …
Gaussian ProcessesA Neural Stochastic Volatility Model
In this paper, we show that the recent integration of statistical models with deep recurrent neural networks provides a new way of formulating volatility (the degree of variation of time series) models that have been wid…
modelTime SeriesTime Series Analysis