Memory and forecasting capacities of nonlinear recurrent networks
The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs makes natural the introduction of the network forecasting capacity, that measures the possibility of forecasting time series values using network states. Generic bounds for memory and forecasting capacities are formulated in terms of the number of neurons of the nonlinear recurrent network and the autocovariance function or the spectral density of the input. These bounds generalize well-known estimates in the literature to a dependent inputs setup. Finally, for the particular case of linear recurrent networks with independent inputs it is proved that the memory capacity is given by the rank of the associated controllability matrix, a fact that has been for a long time assumed to be true without proof by the community.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Uncovering the Functional Roles of Nonlinearity in Memory
Memory and long-range temporal processing are core requirements for sequence modeling tasks across natural language processing, time-series forecasting, speech recognition, and control. While nonlinear recurrence has lon…
speech-recognitionSpeech RecognitionTime Series ForecastingStateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting
Long-horizon multivariate time series forecasting (LTSF) remains challenging due to non-stationarity, regime shifts, and error accumulation. The Variability-Aware Recursive Neural Network (VARNN) is designed to track suc…
Multivariate Time Series ForecastingData-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dy…
Gaussian ProcessesTime SeriesTime Series AnalysisWhat is the best RNN-cell structure for forecasting each time series behavior?
It is unquestionable that time series forecasting is of paramount importance in many fields. The most used machine learning models to address time series forecasting tasks are Recurrent Neural Networks (RNNs). Typically,…
Time SeriesTime Series AnalysisTime Series ForecastingEntanglement-Embedded Recurrent Network Architecture: Tensorized Latent State Propagation and Chaos Forecasting
Chaotic time series forecasting has been far less understood despite its tremendous potential in theory and real-world applications. Traditional statistical/ML methods are inefficient to capture chaos in nonlinear dynami…
Tensor DecompositionTime SeriesTime Series AnalysisTime Series Forecasting