Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces
We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the actor-critic (AC) paradigm with a representation of the mean field distribution via a parameterized score function, which can be efficiently updated in an online fashion, and uses Langevin dynamics to obtain samples from the resulting distribution. The AC agent and the score function are updated iteratively to converge, either to the MFG equilibrium or the MFC optimum for a given mean field problem, depending on the choice of learning rates. A straightforward modification of the algorithm allows us to solve mixed mean field control games (MFCGs). The performance of our algorithm is evaluated using linear-quadratic benchmarks in the asymptotic infinite horizon framework.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Unified Reinforcement Q-Learning for Mean Field Game and Control Problems
We present a Reinforcement Learning (RL) algorithm to solve infinite horizon asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach can be described as a unified two-timescale Mean Field Q-l…
Q-LearningReinforcement Learning (RL)Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learning
We study infinite horizon discounted Mean Field Control (MFC) problems with common noise through the lens of Mean Field Markov Decision Processes (MFMDP). We allow the agents to use actions that are randomized not only a…
General Reinforcement LearningQ-LearningReinforcement LearningReinforcement Learning (RL)Model Free Reinforcement Learning Algorithm for Stationary Mean field Equilibrium for Multiple Types of Agents
We consider a multi-agent Markov strategic interaction over an infinite horizon where agents can be of multiple types. We model the strategic interaction as a mean-field game in the asymptotic limit when the number of ag…
Reinforcement Learning (RL)Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces
We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. Thi…
Reinforcement LearningStationary Cost Nodes in Infinite Horizon LQG-GMFGs
An analysis of infinite horizon linear quadratic Gaussian (LQG) Mean Field Games is given within the general framework of Graphon Mean Field Games (GMFG) on dense infinite graphs (or networks) introduced in Caines and Hu…