Modeling Deep Temporal Dependencies with Recurrent Grammar Cells""
We propose modeling time series by representing the transformations that take a frame at time t to a frame at time t+1. To this end we show how a bi-linear model of transformations, such as a gated autoencoder, can be turned into a recurrent network, by training it to predict future frames from the current one and the inferred transformation using backprop-through-time. We also show how stacking multiple layers of gating units in a recurrent pyramid makes it possible to represent the ”syntax” of complicated time series, and that it can outperform standard recurrent neural networks in terms of prediction accuracy on a variety of tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Recurrent Neural Networks for Forecasting Time Series with Multiple Seasonality: A Comparative Study
This paper compares recurrent neural networks (RNNs) with different types of gated cells for forecasting time series with multiple seasonality. The cells we compare include classical long short term memory (LSTM), gated …
Load ForecastingTime SeriesTime Series AnalysisCells in Multidimensional Recurrent Neural Networks
The transcription of handwritten text on images is one task in machine learning and one solution to solve it is using multi-dimensional recurrent neural networks (MDRNN) with connectionist temporal classification (CTC). …
Efficient keyword spotting using dilated convolutions and gating
We explore the application of end-to-end stateless temporal modeling to small-footprint keyword spotting as opposed to recurrent networks that model long-term temporal dependencies using internal states. We propose a mod…
Keyword SpottingSmall-Footprint Keyword SpottingNon-local Recurrent Neural Memory for Supervised Sequence Modeling
Typical methods for supervised sequence modeling are built upon the recurrent neural networks to capture temporal dependencies. One potential limitation of these methods is that they only model explicitly information int…
Action RecognitionSentiment AnalysisWarming up recurrent neural networks to maximise reachable multistability greatly improves learning
Training recurrent neural networks is known to be difficult when time dependencies become long. In this work, we show that most standard cells only have one stable equilibrium at initialisation, and that learning on task…
Time Series Analysis