Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graph

2022-01-10 · Gangshan Jing, He Bai, Jemin George, Aranya Chakrabortty, Piyush. K. Sharma

Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local communication with its neighbors through a directed learning-induced communication graph, without using any consensus algorithm. A zeroth-order optimization (ZOO) approach based on parameter perturbation is employed to achieve gradient estimation. By comparing with existing ZOO-based RL algorithms, we show that our proposed distributed RL algorithm guarantees high scalability. A distributed resource allocation example is shown to illustrate the effectiveness of our algorithm.

📄 PDF Abstract BibTeX arXiv:2201.04962

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning

2019-07-06 · Yixuan Lin, Kaiqing Zhang, Zhuoran Yang, Zhaoran Wang 외

This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A random…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation

2020-03-21 · Guohui Ding, Joewie J. Koh, Kelly Merckaert, Bram Vanderborght 외

We consider solving a cooperative multi-robot object manipulation task using reinforcement learning (RL). We propose two distributed multi-agent RL approaches: distributed approximate RL (DA-RL), where each agent applies…

Object Q-Learning reinforcement-learning Reinforcement Learning+1

Distributed Differential Graphical Game for Control of Double-Integrator Multi-Agent Systems with Input Delay

2023-10-16 · Hossein B. Jond

This paper studies cooperative control of noncooperative double-integrator multi-agent systems (MASs) with input delay on connected directed graphs in the context of a differential graphical game (DGG). In the distribute…

Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation

2021-03-08 · Abhimanyu Dubey, Alex Pentland

Reinforcement learning in cooperative multi-agent settings has recently advanced significantly in its scope, with applications in cooperative estimation for advertising, dynamic treatment regimes, distributed control, an…

Federated Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Group-Agent Reinforcement Learning

2022-02-10 · Kaiyue Wu, Xiao-jun Zeng

It can largely benefit the reinforcement learning (RL) process of each agent if multiple geographically distributed agents perform their separate RL tasks cooperatively. Different from multi-agent reinforcement learning …