Learning Regularized Monotone Graphon Mean-Field Games
This paper studies two fundamental problems in regularized Graphon Mean-Field Games (GMFGs). First, we establish the existence of a Nash Equilibrium (NE) of any $\lambda$-regularized GMFG (for $\lambda\geq 0$). This result relies on weaker conditions than those in previous works for analyzing both unregularized GMFGs ($\lambda=0$) and $\lambda$-regularized MFGs, which are special cases of GMFGs. Second, we propose provably efficient algorithms to learn the NE in weakly monotone GMFGs, motivated by Lasry and Lions [2007]. Previous literature either only analyzed continuous-time algorithms or required extra conditions to analyze discrete-time algorithms. In contrast, we design a discrete-time algorithm and derive its convergence rate solely under weakly monotone conditions. Furthermore, we develop and analyze the action-value function estimation procedure during the online learning process, which is absent from algorithms for monotone GMFGs. This serves as a sub-module in our optimization algorithm. The efficiency of the designed algorithm is corroborated by empirical evaluations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning Graphon Mean Field Games and Approximate Nash Equilibria
Recent advances at the intersection of dense large graph limits and mean field games have begun to enable the scalable analysis of a broad class of dynamical sequential games with large numbers of agents. So far, results…
Learning Regularized Graphon Mean-Field Games with Unknown Graphons
We design and analyze reinforcement learning algorithms for Graphon Mean-Field Games (GMFGs). In contrast to previous works that require the precise values of the graphons, we aim to learn the Nash Equilibrium (NE) of th…
LQG Graphon Mean Field Games: Analysis via Graphon Invariant Subspaces
This paper studies approximate solutions to large-scale linear quadratic stochastic games with homogeneous nodal dynamics parameters and heterogeneous network couplings within the graphon mean field game framework in [2]…
Hypergraphon Mean Field Games
We propose an approach to modelling large-scale multi-agent dynamical systems allowing interactions among more than just pairs of agents using the theory of mean field games and the notion of hypergraphons, which are obt…
Mean Field Games on Weighted and Directed Graphs via Colored Digraphons
The field of multi-agent reinforcement learning (MARL) has made considerable progress towards controlling challenging multi-agent systems by employing various learning methods. Numerous of these approaches focus on empir…
Multi-agent Reinforcement Learning