Multi-Agent Reinforcement Learning in Stochastic Networked Systems

2020-06-11 · NeurIPS 2021 12 · Yiheng Lin, Guannan Qu, Longbo Huang, Adam Wierman

We study multi-agent reinforcement learning (MARL) in a stochastic network of agents. The objective is to find localized policies that maximize the (discounted) global reward. In general, scalability is a challenge in this setting because the size of the global state/action space can be exponential in the number of agents. Scalable algorithms are only known in cases where dependencies are static, fixed and local, e.g., between neighbors in a fixed, time-invariant underlying graph. In this work, we propose a Scalable Actor Critic framework that applies in settings where the dependencies can be non-local and stochastic, and provide a finite-time error bound that shows how the convergence rate depends on the speed of information spread in the network. Additionally, as a byproduct of our analysis, we obtain novel finite-time convergence results for a general stochastic approximation scheme and for temporal difference learning with state aggregation, which apply beyond the setting of MARL in networked systems.

📄 PDF Abstract BibTeX arXiv:2006.06555

Code (1)

yihenglin97/networked-marl pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning

2025-09-20 · Wei Duan, Jie Lu, Junyu Xuan arxiv

In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained communication over fixed physical graphs. Existing methods often assume static nei…

Multi-agent Reinforcement Learning

Safety Embedded Stochastic Optimal Control of Networked Multi-Agent Systems via Barrier States

2022-10-08 · Lin Song, Pan Zhao, Neng Wan, Naira Hovakimyan

This paper presents a novel approach for achieving safe stochastic optimal control in networked multi-agent systems (MASs). The proposed method incorporates barrier states (BaSs) into the system dynamics to embed safety …

Learning Predictive Communication by Imagination in Networked System Control

2021-01-01 · Yali Du, Yifan Zhao, Meng Fang, Jun Wang 외

Dealing with multi-agent control in networked systems is one of the biggest challenges in Reinforcement Learning (RL) and limited success has been presented compared to recent deep reinforcement learning in single-agent …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning

2020-10-15 · NeurIPS 2020 12 · Arnu Pretorius, Scott Cameron, Elan van Biljon, Tom Makkink 외

Multi-agent reinforcement learning has recently shown great promise as an approach to networked system control. Arguably, one of the most difficult and important tasks for which large scale networked system control is ap…

Management Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

General sum stochastic games with networked information flows

2022-05-05 · Sarah H. Q. Li, Lillian J. Ratliff, Peeyush Kumar

Inspired by applications such as supply chain management, epidemics, and social networks, we formulate a stochastic game model that addresses three key features common across these domains: 1) network-structured player i…

Decision Making Management Multi-agent Reinforcement Learning reinforcement-learning+1