Unified Reinforcement Q-Learning for Mean Field Game and Control Problems
We present a Reinforcement Learning (RL) algorithm to solve infinite horizon asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach can be described as a unified two-timescale Mean Field Q-learning: The \emph{same} algorithm can learn either the MFG or the MFC solution by simply tuning the ratio of two learning parameters. The algorithm is in discrete time and space where the agent not only provides an action to the environment but also a distribution of the state in order to take into account the mean field feature of the problem. Importantly, we assume that the agent can not observe the population's distribution and needs to estimate it in a model-free manner. The asymptotic MFG and MFC problems are also presented in continuous time and space, and compared with classical (non-asymptotic or stationary) MFG and MFC problems. They lead to explicit solutions in the linear-quadratic (LQ) case that are used as benchmarks for the results of our algorithm.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces
We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach…
Deep Reinforcement LearningReinforcement Learning (RL)Reinforcement Learning for Mean Field Games, with Applications to Economics
Mean field games (MFG) and mean field control problems (MFC) are frameworks to study Nash equilibria or social optima in games with a continuum of agents. These problems can be used to approximate competitive or cooperat…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Unified continuous-time q-learning for mean-field game and mean-field control problems
This paper studies the continuous-time q-learning in mean-field jump-diffusion models when the population distribution is not directly observable. We propose the integrated q-function in decoupled form (decoupled Iq-func…
Q-LearningTheoretical understanding of adversarial reinforcement learning via mean-field optimal control
Adversarial reinforcement learning has been shown promising in solving games in adversarial environments, while the theoretical understanding is still premature. This paper theoretically analyses the convergence and gene…
Generalization Boundsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces
We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. Thi…
Reinforcement Learning