Actor critic learning algorithms for mean-field control with moment neural networks

2023-09-08 · Huyên Pham, Xavier Warin

We develop a new policy gradient and actor-critic algorithm for solving mean-field control problems within a continuous time reinforcement learning setting. Our approach leverages a gradient-based representation of the value function, employing parametrized randomized policies. The learning for both the actor (policy) and critic (value function) is facilitated by a class of moment neural network functions on the Wasserstein space of probability measures, and the key feature is to sample directly trajectories of distributions. A central challenge addressed in this study pertains to the computational treatment of an operator specific to the mean-field framework. To illustrate the effectiveness of our methods, we provide a comprehensive set of numerical results. These encompass diverse examples, including multi-dimensional settings and nonlinear quadratic mean-field control problems with controlled volatility.

📄 PDF Abstract BibTeX arXiv:2309.04317

Code (0)

등록된 구현이 없습니다.

Actor-Critic learning for mean-field control in continuous time

2023-03-13 · Noufel Frikha, Maximilien Germain, Mathieu Laurière, Huyên Pham 외

We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering randomised policies with entropy regularisation, we derive a gradient expectation representation of t…

reinforcement-learning Reinforcement Learning (RL)

Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms

2026-04-30 · Zhenjie Ren, Xiaoli Wei, Xiang Yu, Xun Yu Zhou arxiv

This paper is a continuation work of Ren et al. (2026) aiming to further devise q-learning algorithms for mean-field control (MFC) with controlled common noise. Based on the relaxed control formulation, we first establis…

Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces

2025-11-10 · Jean-Pierre Fouque, Mathieu Laurière, Mengrui Zhang arxiv

We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. Thi…

Reinforcement Learning

Learning Mean-Field Games through Mean-Field Actor-Critic Flow

2025-10-14 · Mo Zhou, Haosheng Zhou, Ruimeng Hu arxiv

We propose the Mean-Field Actor-Critic (MFAC) flow, a continuous-time learning dynamics for solving mean-field games (MFGs), combining techniques from reinforcement learning and optimal transport. The MFAC framework join…

Reinforcement Learning

Mean Actor Critic

2017-09-01 · Cameron Allen, Kavosh Asadi, Melrose Roderick, Abdel-rahman Mohamed 외

We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action continuous-state reinforcement learning. MAC is a policy gradient algorithm that uses the agent's explicit representation of all action values to e…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)