홈 › Papers

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

2020-10-31 · Dong-Ki Kim, Miao Liu, Matthew Riemer, Chuangchuang Sun, Marwa Abdulhai, Golnaz Habibi, Sebastian Lopez-Cot, Gerald Tesauro, Jonathan P. How

A fundamental challenge in multiagent reinforcement learning is to learn beneficial behaviors in a shared environment with other simultaneously learning agents. In particular, each agent perceives the environment as effectively non-stationary due to the changing policies of other agents. Moreover, each agent is itself constantly learning, leading to natural non-stationarity in the distribution of experiences encountered. In this paper, we propose a novel meta-multiagent policy gradient theorem that directly accounts for the non-stationary policy dynamics inherent to multiagent learning settings. This is achieved by modeling our gradient updates to consider both an agent's own non-stationary policy dynamics and the non-stationary policy dynamics of other agents in the environment. We show that our theoretically grounded approach provides a general solution to the multiagent learning problem, which inherently comprises all key aspects of previous state of the art approaches on this topic. We test our method on a diverse suite of multiagent benchmarks and demonstrate a more efficient ability to adapt to new agents as they learn than baseline methods across the full spectrum of mixed incentive, competitive, and cooperative domains.

📄 PDF Abstract BibTeX arXiv:2011.00382

Code (1)

dkkim93/meta-mapg 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning (RL)

Actor-Critic Policy Optimization in Partially Observable Multiagent Environments

2018-10-21 · NeurIPS 2018 12 · Sriram Srinivasan, Marc Lanctot, Vinicius Zambaldi, Julien Perolat 외

Optimization of parameterized policies for reinforcement learning (RL) is an important and challenging problem in artificial intelligence. Among the most common approaches are algorithms based on gradient ascent of a sco…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

COLLABORATIVE MULTIAGENT REINFORCEMENT LEARNING IN HOMOGENEOUS SWARMS

2018-09-27 · Arbaaz Khan, Clark Zhang, Vijay Kumar, Alejandro Ribeiro

A deep reinforcement learning solution is developed for a collaborative multiagent system. Individual agents choose actions in response to the state of the environment, their own state, and possibly partial information a…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multiagent Soft Q-Learning

2018-04-25 · Ermo Wei, Drew Wicke, David Freelan, Sean Luke

Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic p…

Policy Gradient Methods Q-Learning reinforcement-learning Reinforcement Learning+1

Policy Distillation and Value Matching in Multiagent Reinforcement Learning

2019-03-15 · Samir Wadhwania, Dong-Ki Kim, Shayegan Omidshafiei, Jonathan P. How

Multiagent reinforcement learning algorithms (MARL) have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information betwe…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Difference Rewards Policy Gradients

2020-12-21 · Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani

Policy gradient methods have become one of the most popular classes of algorithms for multi-agent reinforcement learning. A key challenge, however, that is not addressed by many of these methods is multi-agent credit ass…

counterfactual Multi-agent Reinforcement Learning Policy Gradient Methods Reinforcement Learning (RL)