Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward

2020-06-11 · NeurIPS 2020 12 · Guannan Qu, Yiheng Lin, Adam Wierman, Na Li

It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents. In this paper, we identify a rich class of networked MARL problems where the model exhibits a local dependence structure that allows it to be solved in a scalable manner. Specifically, we propose a Scalable Actor-Critic (SAC) method that can learn a near optimal localized policy for optimizing the average reward with complexity scaling with the state-action space size of local neighborhoods, as opposed to the entire network. Our result centers around identifying and exploiting an exponential decay property that ensures the effect of agents on each other decays exponentially fast in their graph distance.

📄 PDF Abstract BibTeX arXiv:2006.06626

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Exponential Decay Exponential Decay is a learning rate schedule where we decay the learning rate with more iterations using an exponential function: $$ \text{lr} =…

Multi-Agent Reinforcement Learning in Stochastic Networked Systems

2020-06-11 · NeurIPS 2021 12 · Yiheng Lin, Guannan Qu, Longbo Huang, Adam Wierman

We study multi-agent reinforcement learning (MARL) in a stochastic network of agents. The objective is to find localized policies that maximize the (discounted) global reward. In general, scalability is a challenge in th…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Scalable Reinforcement Learning of Localized Policies for Multi-Agent Networked Systems

2020-06-08 · L4DC 2020 6 · Guannan Qu, Adam Wierman, Na Li

We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is…

reinforcement-learning Reinforcement Learning (RL)

Scalable Reinforcement Learning for Multi-Agent Networked Systems

2019-12-05 · Guannan Qu, Adam Wierman, Na Li

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Predictive Communication by Imagination in Networked System Control

2021-01-01 · Yali Du, Yifan Zhao, Meng Fang, Jun Wang 외

Dealing with multi-agent control in networked systems is one of the biggest challenges in Reinforcement Learning (RL) and limited success has been presented compared to recent deep reinforcement learning in single-agent …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning

2020-10-15 · NeurIPS 2020 12 · Arnu Pretorius, Scott Cameron, Elan van Biljon, Tom Makkink 외

Multi-agent reinforcement learning has recently shown great promise as an approach to networked system control. Arguably, one of the most difficult and important tasks for which large scale networked system control is ap…

Management Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1