A Dynamically Controlled Recurrent Neural Network for Modeling Dynamical Systems

2019-10-31 · Yiwei Fu, Samer Saab Jr, Asok Ray, Michael Hauser

This work proposes a novel neural network architecture, called the Dynamically Controlled Recurrent Neural Network (DCRNN), specifically designed to model dynamical systems that are governed by ordinary differential equations (ODEs). The current state vectors of these types of dynamical systems only depend on their state-space models, along with the respective inputs and initial conditions. Long Short-Term Memory (LSTM) networks, which have proven to be very effective for memory-based tasks, may fail to model physical processes as they tend to memorize, rather than learn how to capture the information on the underlying dynamics. The proposed DCRNN includes learnable skip-connections across previously hidden states, and introduces a regularization term in the loss function by relying on Lyapunov stability theory. The regularizer enables the placement of eigenvalues of the transfer function induced by the DCRNN to desired values, thereby acting as an internal controller for the hidden state trajectory. The results show that, for forecasting a chaotic dynamical system, the DCRNN outperforms the LSTM in $100$ out of $100$ randomized experiments by reducing the mean squared error of the LSTM's forecasting by $80.0\% \pm 3.0\%$.

📄 PDF Abstract BibTeX arXiv:1911.00089

Code (0)

등록된 구현이 없습니다.

Tasks

State Space Models

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

InputDSA: Demixing then Comparing Recurrent and Externally Driven Dynamics

2025-10-29 · Ann Huang, Mitchell Ostrow, Satpreet H. Singh, Leo Kozachkov 외 arxiv

In control problems and basic scientific modeling, it is important to compare observations with dynamical simulations. For example, comparing two neural systems can shed light on the nature of emergent computations in th…

Reinforcement Learning

Dynamically Computing Adversarial Perturbations for Recurrent Neural Networks

2020-09-07 · Shankar A. Deka, Dušan M. Stipanović, Claire J. Tomlin

Convolutional and recurrent neural networks have been widely employed to achieve state-of-the-art performance on classification tasks. However, it has also been noted that these networks can be manipulated adversarially …

Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels

2023-11-12 · NeurIPS 2023 11

Humans solving algorithmic (or) reasoning problems typically exhibit solution times that grow as a function of problem difficulty. Adaptive recurrent neural networks have been shown to exhibit this property for various l…

Pathfinder Visual Reasoning Zero-shot Generalization

Neural Speed Reading via Skim-RNN

2017-11-06 · ICLR 2018 1 · Minjoon Seo, Sewon Min, Ali Farhadi, Hannaneh Hajishirzi

Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. S…

CPU

Transformer-Based Neural Text Generation with Syntactic Guidance

2020-10-05 · Yinghao Li, Rui Feng, Isaac Rehg, Chao Zhang

We study the problem of using (partial) constituency parse trees as syntactic guidance for controlled text generation. Existing approaches to this problem use recurrent structures, which not only suffer from the long-ter…

Decoder Text Generation