Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation

2021-03-08 · Abhimanyu Dubey, Alex Pentland

Reinforcement learning in cooperative multi-agent settings has recently advanced significantly in its scope, with applications in cooperative estimation for advertising, dynamic treatment regimes, distributed control, and federated learning. In this paper, we discuss the problem of cooperative multi-agent RL with function approximation, where a group of agents communicates with each other to jointly solve an episodic MDP. We demonstrate that via careful message-passing and cooperative value iteration, it is possible to achieve near-optimal no-regret learning even with a fixed constant communication budget. Next, we demonstrate that even in heterogeneous cooperative settings, it is possible to achieve Pareto-optimal no-regret learning with limited communication. Our work generalizes several ideas from the multi-agent contextual and multi-armed bandit literature to MDPs and reinforcement learning.

📄 PDF Abstract BibTeX arXiv:2103.04972

Code (0)

등록된 구현이 없습니다.

Tasks

Federated Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation

2023-05-10 · Yifei Min, Jiafan He, Tianhao Wang, Quanquan Gu

We study multi-agent reinforcement learning in the setting of episodic Markov decision processes, where multiple agents cooperate via communication through a central server. We propose a provably efficient algorithm base…

Multi-agent Reinforcement Learning reinforcement-learning

Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning

2023-05-08 · Yulai Zhao, Zhuoran Yang, Zhaoran Wang, Jason D. Lee

Policy optimization methods with function approximation are widely used in multi-agent reinforcement learning. However, it remains elusive how to design such algorithms with statistical guarantees. Leveraging a multi-age…

LEMMA Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Cooperative and Competitive Biases for Multi-Agent Reinforcement Learning

2021-01-18 · Heechang Ryu, Hayong Shin, Jinkyoo Park

Training a multi-agent reinforcement learning (MARL) algorithm is more challenging than training a single-agent reinforcement learning algorithm, because the result of a multi-agent task strongly depends on the complex i…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

2022-06-15 · Wei Fu, Chao Yu, Zelai Xu, Jiaqi Yang 외

Many advances in cooperative multi-agent reinforcement learning (MARL) are based on two common design principles: value decomposition and parameter sharing. A typical MARL algorithm of this fashion decomposes a centraliz…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Specification-Guided Learning of Nash Equilibria with High Social Welfare

2022-06-06 · Kishor Jothimurugan, Suguman Bansal, Osbert Bastani, Rajeev Alur

Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement le…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Intensity Prediction