Slow Transition to Low-Dimensional Chaos in Heavy-Tailed Recurrent Neural Networks
Growing evidence suggests that synaptic weights in the brain follow heavy-tailed distributions, yet most theoretical analyses of recurrent neural networks (RNNs) assume Gaussian connectivity. We systematically study the activity of RNNs with random weights drawn from biologically plausible L\'evy alpha-stable distributions. While mean-field theory for the infinite system predicts that the quiescent state is always unstable -- implying ubiquitous chaos -- our finite-size analysis reveals a sharp transition between quiescent and chaotic dynamics. We theoretically predict the gain at which the system transitions from quiescent to chaotic dynamics, and validate it through simulations. Compared to Gaussian networks, heavy-tailed RNNs exhibit a broader parameter regime near the edge of chaos, namely a slow transition to chaos. However, this robustness comes with a tradeoff: heavier tails reduce the Lyapunov dimension of the attractor, indicating lower effective dimensionality. Our results reveal a biologically aligned tradeoff between the robustness of dynamics near the edge of chaos and the richness of high-dimensional neural activity. By analytically characterizing the transition point in finite-size networks -- where mean-field theory breaks down -- we provide a tractable framework for understanding dynamics in realistically sized, heavy-tailed neural circuits.
Code (1)
Similar Papers 제목 키워드 기반
Robust Sparse Reduced Rank Regression in High Dimensions
We propose robust sparse reduced rank regression for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed method is based on a convex relaxation of a rank- and sparsity-constrain…
regressionVocal Bursts Intensity PredictionHigh Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantization
In this paper, we propose a uniformly dithered 1-bit quantization scheme for high-dimensional statistical estimation. The scheme contains truncation, dithering, and quantization as typical steps. As canonical examples, t…
compressed sensingLow-Rank Matrix CompletionMatrix CompletionQuantization+1High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality
We investigate the high-dimensional properties of robust regression estimators in the presence of heavy-tailed contamination of both the covariates and response functions. In particular, we provide a sharp asymptotic cha…
regressionThree dimensional chaos game representation of protein sequences
A new three dimensional approach to the chaos game representation of protein sequences is explored in this thesis. The basics of DNA, the synthesis of proteins from DNA, protein structure and functionality and sequence a…
No-Regret Reinforcement Learning with Heavy-Tailed Rewards
Reinforcement learning algorithms typically assume rewards to be sampled from light-tailed distributions, such as Gaussian or bounded. However, a wide variety of real-world systems generate rewards that follow heavy-tail…
Deep Reinforcement LearningQ-Learningreinforcement-learningReinforcement Learning+1