On Distributional Reinforcement Learning in Chaotic Dynamical Systems
Chaotic dynamical systems pose a fundamental challenge for Reinforcement Learning (RL): exponential sensitivity to initial conditions induces high-variance bootstrap targets and poorly conditioned gradient updates. Chaotic dynamics arise across scientific and engineering domains, from fluid flows and climate systems to multi-agent systems, where reliable learning is highly desirable. Standard RL methods optimise expected returns through scalar value functions, implicitly averaging over diverging trajectories and entangling trajectory level instability with the learning objective. We show that under mild statistical stability assumptions, the return distribution evolves more regularly than individual trajectories when measured under the $1$-Wasserstein metric, yielding a smoother distributional Bellman objective. By aligning optimisation with this measure level structure, distributional RL provides better conditioned learning. We offer a principled explanation for the advantages of distributional methods in chaotic systems and the geometries of RL objectives under chaos.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Restoring Chaos Using Deep Reinforcement Learning
A catastrophic bifurcation in non-linear dynamical systems, called crisis, often leads to their convergence to an undesirable non-chaotic state after some initial chaotic transients. Preventing such behavior has proved t…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Adaptive Diffusion Posterior Sampling for Data and Model Fusion of Complex Nonlinear Dynamical Systems
High-fidelity numerical simulations of chaotic, high dimensional nonlinear dynamical systems are computationally expensive, necessitating the development of efficient surrogate models. Most surrogate models for such syst…
Analysis of chaotic dynamical systems with autoencoders
We focus on chaotic dynamical systems and analyze their time series with the use of autoencoders, i.e., configurations of neural networks that map identical output to input. This analysis results in the determination of …
Time SeriesTime Series AnalysisDivide And Conquer: Learning Chaotic Dynamical Systems With Multistep Penalty Neural Ordinary Differential Equations
Forecasting high-dimensional dynamical systems is a fundamental challenge in various fields, such as geosciences and engineering. Neural Ordinary Differential Equations (NODEs), which combine the power of neural networks…
Stochastic generative methods for stable and accurate closure modeling of chaotic dynamical systems
Traditional deterministic subgrid-scale (SGS) models are often dissipative and unstable, especially in regions of chaotic and turbulent flow. Ongoing work in climate science and ocean modeling motivates the use of stocha…