paper-with-me

홈 › Papers

Between Amnesia and Chaos: A Memory Stability Expressivity Trilemma for Trainable Dissipative Oscillator Networks

2026-06-07 · Caleb Munigety arxiv

Physical reservoir computing harnesses nonlinear mechanical dynamics but, by convention, freezes the substrate and trains only a linear readout, presuming the substrate is not usefully trainable. We revisit that premise for networks of nonlinear oscillators whose mass, damping, and stiffness are learned end-to-end through a symplectic integrator. Our central result is a trilemma: memory horizon, gradient stability, and dynamical expressivity cannot be simultaneously maximized, because all three are governed by the damping. The backward gradient decays at a rate set by the damping, capping how far back credit can propagate, while forward sensitivities grow exponentially in the largest Lyapunov exponent, so usable gradients require damping above a stability floor. Since the Lyapunov exponent falls as damping rises while the memory ceiling falls as the horizon grows, stable training is confined to a band that contracts with horizon and closes at a critical point. We test every step on a twenty-oscillator network. A damping sweep finds the largest Lyapunov exponent monotone and crossing zero at a well-defined stability floor, confirming the theorem's key assumption. A compute-matched comparison of learned versus frozen substrate on delayed recall across nine horizons shows the learned substrate dominating at short horizons and the advantage closing and reversing near a horizon of eleven steps, the predicted signature of band closure; trained models settle near the stability floor, seeking the edge of chaos unprompted. The analytic ceiling overestimates the empirical crossover roughly fivefold, a gap between detectable and learnable gradient that we report rather than tune away. The contribution is a confirmed account of when training a physical substrate beats freezing it.

📄 PDF Abstract BibTeX arXiv:2606.09929

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective

2023-10-10 · Takeshi Koshizuka, Masahiro Fujisawa, Yusuke Tanaka, Issei Sato

In this paper, we explores the expressivity and trainability of the Fourier Neural Operator (FNO). We establish a mean-field theory for the FNO, analyzing the behavior of the random FNO from an edge of chaos perspective.…

A consequence of failed sequential learning: A computational account of developmental amnesia

2026-02-13 · Qi Zhang arxiv

Developmental amnesia, featured with severely impaired episodic memory and almost normal semantic memory, has been discovered to occur in children with hippocampal atrophy. This unique combination of characteristics seem…

Optimal sequence memory in driven random networks

2017-09-22

Autonomous randomly coupled neural networks display a transition to chaos at a critical coupling strength. We here investigate the effect of a time-varying input on the onset of chaos and the resulting consequences for i…

The Expressivity and Training of Deep Neural Networks: toward the Edge of Chaos?

2019-10-11 · Gege Zhang, Gangwei Li, Ningwei Shen, Weidong Zhang

Expressivity is one of the most significant issues in assessing neural networks. In this paper, we provide a quantitative analysis of the expressivity for the deep neural network (DNN) from its dynamic model, where the H…

image-classificationImage ClassificationTime SeriesTime Series Analysis+1

Exponential expressivity in deep neural networks through transient chaos

2016-06-16 · NeurIPS 2016 12 · Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein 외

We combine Riemannian geometry with the mean field theory of high dimensional chaos to study the nature of signal propagation in generic, deep neural networks with random weights. Our results reveal an order-to-chaos exp…