Scalable Neighborhood-Based Multi-Agent Actor-Critic

2026-04-20 · Tim Goppelsroeder, Rasmus Jensen arxiv

We propose MADDPG-K, a scalable extension to Multi-Agent Deep Deterministic Policy Gradient (MADDPG) that addresses the computational limitations of centralized critic approaches. Centralized critics, which condition on the observations and actions of all agents, have demonstrated significant performance gains in cooperative and competitive multi-agent settings. However, their critic networks grow linearly in input size with the number of agents, making them increasingly expensive to train at scale. MADDPG-K mitigates this by restricting each agent's critic to the $k$ closest agents under a chosen metric which in our case is Euclidean distance. This ensures a constant-size critic input regardless of the total agent count. We analyze the complexity of this approach, showing that the quadratic cost it retains arises from cheap scalar distance computations rather than the expensive neural network matrix multiplications that bottleneck standard MADDPG. We validate our method empirically across cooperative and adversarial environments from the Multi-Particle Environment suite, demonstrating competitive or superior performance compared to MADDPG, faster convergence in cooperative settings, and better runtime scaling as the number of agents grows. Our code is available at https://github.com/TimGop/MADDPG-K .

📄 PDF Abstract BibTeX arXiv:2604.18190

Code (0)

등록된 구현이 없습니다.

Causality Meets Locality: Provably Generalizable and Scalable Policy Learning for Networked Systems

2025-10-24 · Hao Liang, Shuqing Shi, Yudi Zhang, Biwei Huang 외 arxiv

Large-scale networked systems, such as traffic, power, and wireless grids, challenge reinforcement-learning agents with both scale and environment shifts. To address these challenges, we propose GSAC (Generalizable and S…

Representation Learning Domain Generalization

Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward

2020-06-11 · NeurIPS 2020 12 · Guannan Qu, Yiheng Lin, Adam Wierman, Na Li

It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games

2023-03-08 · Zhaoyi Zhou, Zaiwei Chen, Yiheng Lin, Adam Wierman

We introduce a class of networked Markov potential games in which agents are associated with nodes in a network. Each agent has its own local potential function, and the reward of each agent depends only on the states an…

Mean-Field Control on Sparse Graphs: From Local Limits to GNNs via Neighborhood Distributions

2026-01-29 · Tobias Schmidt, Kai Cui arxiv

Mean-field control (MFC) offers a scalable solution to the curse of dimensionality in multi-agent systems but traditionally hinges on the restrictive assumption of exchangeability via dense, all-to-all interactions. In t…

Reinforcement Learning

Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation

2022-11-03 · Siddharth Nayak, Kenneth Choi, Wenqi Ding, Sydney Dolan 외

We consider the problem of multi-agent navigation and collision avoidance when observations are limited to the local neighborhood of each agent. We propose InforMARL, a novel architecture for multi-agent reinforcement le…

Collision Avoidance Graph Neural Network Multi-agent Reinforcement Learning reinforcement-learning+2