Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functions

2026-04-06 · Daniel Bloch arxiv

This paper introduces Anticipatory Reinforcement Learning (ARL), a novel framework designed to bridge the gap between non-Markovian decision processes and classical reinforcement learning architectures, specifically under the constraint of a single observed trajectory. In environments characterised by jump-diffusions and structural breaks, traditional state-based methods often fail to capture the essential path-dependent geometry required for accurate foresight. We resolve this by lifting the state space into a signature-augmented manifold, where the history of the process is embedded as a dynamical coordinate. By utilising a self-consistent field approach, the agent maintains an anticipated proxy of the future path-law, allowing for a deterministic evaluation of expected returns. This transition from stochastic branching to a single-pass linear evaluation significantly reduces computational complexity and variance. We prove that this framework preserves fundamental contraction properties and ensures stable generalisation even in the presence of heavy-tailed noise. Our results demonstrate that by grounding reinforcement learning in the topological features of path-space, agents can achieve proactive risk management and superior policy stability in highly volatile, continuous-time environments.

📄 PDF Abstract BibTeX arXiv:2604.04662

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Generative Path-Law Jump-Diffusion: Sequential MMD-Gradient Flows and Generalisation Bounds in Marcus-Signature RKHS

2026-04-06 · Daniel Bloch arxiv

This paper introduces a novel generative framework for synthesising forward-looking, càdlàg stochastic trajectories that are sequentially consistent with time-evolving path-law proxies, thereby incorporating anticipated …

Computational Efficiency

Learning Generative Dynamics with Soft Law Constraints: A McKean-Vlasov FBSDE Approach

2026-05-09 · Samer El Boustany, Samy Mekkaoui, Yadh Hafsi, Alexandre Alouadi 외 arxiv

We propose a generative framework for learning stochastic dynamics from endpoint and intermediate distributional observations. The method formulates generation as a McKean-Vlasov control problem in which terminal and tim…

Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learning

2026-08-13 · Zijie Cheng, Yang Peng, Zhihua Zhang arxiv

In this paper, we study how to perform statistical inference for quantile temporal difference learning (QTD) in distributional reinforcement learning. Assuming access to a generative model, we first establish functional …

Reinforcement Learning

On solutions of the distributional Bellman equation

2022-01-31 · Julian Gerstenberg, Ralph Neininger, Denis Spiegel

In distributional reinforcement learning not only expected returns but the complete return distributions of a policy are taken into account. The return distribution for a fixed policy is given as the solution of an assoc…

Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model

2024-02-12 · Mark Rowland, Li Kevin Wenliang, Rémi Munos, Clare Lyle 외

We propose a new algorithm for model-based distributional reinforcement learning (RL), and prove that it is minimax-optimal for approximating return distributions with a generative model (up to logarithmic factors), reso…