Mathematical methods of reinforcement learning
Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory. This survey organizes the mathematical structures that underpin the design and analysis of modern algorithms in RL. We begin from Markov decision processes (MDPs) and the Bellman operators, emphasizing contraction mappings, monotonicity, and fixed-point theory that yield convergence guarantees and rates for value and policy iteration, and temporal-difference schemes. We then develop the optimization perspective: stochastic approximation and martingale methods, convex duality and the role of regularization linking mirror/proximal methods. Function approximation is treated through linear and non-linear settings, covering stabilization, error decomposition, and sample-complexity via concentration inequalities for dependent data and mixing processes. We further cover off-policy evaluation/learning, constrained RL and constrained MDPs (CMDPs). Throughout we unify algorithmic templates under common operator and variational lenses, highlighting both finite-sample bounds and asymptotic results. Our presentation is intended to provide a unified mathematical entry point for researchers in probability, optimization, and statistics interested in reinforcement learning.
Code (1)
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
MathDSL: A Domain-Specific Language for Concise Mathematical Solutions Via Program Synthesis
We present MathDSL, a Domain-Specific Language (DSL) for mathematical equation solving, which, when deployed in program synthesis models, outperforms state-of-the-art reinforcement-learning-based methods. We also introdu…
Program Synthesisreinforcement-learningReinforcement LearningVCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
Policy-based reinforcement learning currently plays an important role in improving LLMs on mathematical reasoning tasks. However, existing rollout-based reinforcement learning methods (GRPO, DAPO, GSPO, etc.) fail to exp…
Reinforcement LearningMathematical ReasoningDSFT: Inspiring Diffusion Large Language Models to Comprehend Mathematical and Logical Patterns
Diffusion large language models (dLLMs) have emerged as a new architecture following auto regressive models. Their denoising process offers a powerful generative advantage, but they present significant challenges in lear…
Reinforcement LearningGeneral KnowledgeMean Field Reinforcement Learning
This monograph provides an introduction to mean field reinforcement learning through the lens of Markov decision processes arising from large-population stochastic control with mean field interactions and common noise. S…
Multi-agent Reinforcement LearningAdaptive Control of an Inverted Pendulum by a Reinforcement Learning-based LQR Method
Inverted pendulums constitute one of the popular systems for benchmarking control algorithms. Several methods have been proposed for the control of this system, the majority of which rely on the availability of a mathema…
BenchmarkingReinforcement Learning (RL)