Optimal sequence memory in driven random networks
Autonomous randomly coupled neural networks display a transition to chaos at a critical coupling strength. We here investigate the effect of a time-varying input on the onset of chaos and the resulting consequences for information processing. Dynamic mean-field theory yields the statistics of the activity, the maximum Lyapunov exponent, and the memory capacity of the network. We find an exact condition that determines the transition from stable to chaotic dynamics and the sequential memory capacity in closed form. The input suppresses chaos by a dynamic mechanism, shifting the transition to significantly larger coupling strengths than predicted by local stability analysis. Beyond linear stability, a regime of coexistent locally expansive, but non-chaotic dynamics emerges that optimizes the capacity of the network to store sequential input.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Optimal short-term memory before the edge of chaos in driven random recurrent networks
The ability of discrete-time nonlinear recurrent neural networks to store time-varying small input signals is investigated by mean-field theory. The combination of a small input strength and mean-field assumptions makes …
Optimal Multiple Stopping with Negative Discount Rate and Random Refraction Times under Levy Models
This paper studies a class of optimal multiple stopping problems driven by L\'evy processes. Our model allows for a negative effective discount rate, which arises in a number of financial applications, including stock lo…
Short Term Memory Capacity in Networks via the Restricted Isometry Property
Cortical networks are hypothesized to rely on transient network activity to support short term memory (STM). In this paper we study the capacity of randomly connected recurrent linear networks for performing STM when the…
compressed sensingLow-memory stochastic backpropagation with multi-channel randomized trace estimation
Thanks to the combination of state-of-the-art accelerators and highly optimized open software frameworks, there has been tremendous progress in the performance of deep neural networks. While these developments have been …
Semantic SegmentationA Neuromorphic Model of Learning Meaningful Sequences with Long-Term Memory
Learning meaningful sentences is different from learning a random set of words. When humans understand the meaning, the learning occurs relatively quickly. What mechanisms enable this to happen? In this paper, we examine…