Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning

2024-12-01 · Emile Anand, Ishani Karmarkar, Guannan Qu

Designing efficient algorithms for multi-agent reinforcement learning (MARL) is fundamentally challenging because the size of the joint state and action spaces grows exponentially in the number of agents. These difficulties are exacerbated when balancing sequential global decision-making with local agent interactions. In this work, we propose a new algorithm $\texttt{SUBSAMPLE-MFQ}$ ($\textbf{Subsample}$-$\textbf{M}$ean-$\textbf{F}$ield-$\textbf{Q}$-learning) and a decentralized randomized policy for a system with $n$ agents. For any $k\leq n$, our algorithm learns a policy for the system in time polynomial in $k$. We prove that this learned policy converges to the optimal policy on the order of $\tilde{O}(1/\sqrt{k})$ as the number of subsampled agents $k$ increases. In particular, this bound is independent of the number of agents $n$.

📄 PDF Abstract BibTeX arXiv:2412.00661

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Multi-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning

Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning

2026-02-18 · Emile Anand, Richard Hoffmann, Sarah Liaw, Adam Wierman arxiv

Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mea…

Multi-agent Reinforcement Learning

Learning Approximate Nash Equilibria in Cooperative Multi-Agent Reinforcement Learning via Mean-Field Subsampling

2026-03-04 · Emile Anand, Ishani Karmarkar arxiv

Many large-scale platforms and networked control systems have a centralized decision maker interacting with a massive population of agents under strict observability constraints. Motivated by such applications, we study …

Multi-agent Reinforcement Learning

Equilibrium pricing of securities in the co-presence of cooperative and non-cooperative populations

2022-09-26 · Masaaki Fujii

In this work, we develop an equilibrium model for price formation of securities in a market composed of two populations of different types: the first one consists of cooperative agents, while the other one consists of no…

Individual-Level Inverse Reinforcement Learning for Mean Field Games

2022-02-13 · Yang Chen, Libo Zhang, Jiamou Liu, Shuyue Hu

The recent mean field game (MFG) formalism has enabled the application of inverse reinforcement learning (IRL) methods in large-scale multi-agent systems, with the goal of inferring reward signals that can explain demons…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Causal Mean Field Multi-Agent Reinforcement Learning

2025-02-20 · Hao Ma, Zhiqiang Pu, Yi Pan, Boyin Liu 외

Scalability remains a challenge in multi-agent reinforcement learning and is currently under active research. A framework named mean-field reinforcement learning (MFRL) could alleviate the scalability problem by employin…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning