Asynchronous, Option-Based Multi-Agent Policy Gradient: A Conditional Reasoning Approach

2022-03-29 · Xubo Lyu, Amin Banitalebi-Dehkordi, Mo Chen, Yong Zhang

Cooperative multi-agent problems often require coordination between agents, which can be achieved through a centralized policy that considers the global state. Multi-agent policy gradient (MAPG) methods are commonly used to learn such policies, but they are often limited to problems with low-level action spaces. In complex problems with large state and action spaces, it is advantageous to extend MAPG methods to use higher-level actions, also known as options, to improve the policy search efficiency. However, multi-robot option executions are often asynchronous, that is, agents may select and complete their options at different time steps. This makes it difficult for MAPG methods to derive a centralized policy and evaluate its gradient, as centralized policy always select new options at the same time. In this work, we propose a novel, conditional reasoning approach to address this problem and demonstrate its effectiveness on representative option-based multi-agent cooperative tasks through empirical validation. Find code and videos at: \href{https://sites.google.com/view/mahrlsupp/}{https://sites.google.com/view/mahrlsupp/}

📄 PDF Abstract BibTeX arXiv:2203.15925

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning Multi-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning Reinforcement Learning (RL)

Asynchronous Coagent Networks

2019-02-15 · ICML 2020 1 · James E. Kostas, Chris Nota, Philip S. Thomas

Coagent policy gradient algorithms (CPGAs) are reinforcement learning algorithms for training a class of stochastic neural networks called coagent networks. In this work, we prove that CPGAs converge to locally optimal p…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Asynchronous Multi-Agent Actor-Critic with Macro-Actions

2021-09-29 · Yuchen Xiao, Weihao Tan, Christopher Amato

Many realistic multi-agent problems naturally require agents to be capable of performing asynchronously without waiting for other agents to terminate (e.g., multi-robot domains). Such problems can be modeled as Macro-Act…

Decision Making Policy Gradient Methods

Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning

2022-09-20 · Yuchen Xiao, Weihao Tan, Christopher Amato

Synchronizing decisions across multiple agents in realistic settings is problematic since it requires agents to wait for other agents to terminate and communicate about termination reliably. Ideally, agents should learn …

Decision Making Multi-agent Reinforcement Learning Policy Gradient Methods reinforcement-learning+2

Coagent Networks Revisited

2020-01-28 · Modjtaba Shokrian Zini, Mohammad Pedramfar, Matthew Riemer, Ahmadreza Moradipari 외

Coagent networks formalize the concept of arbitrary networks of stochastic agents that collaborate to take actions in a reinforcement learning environment. Prominent examples of coagent networks in action include approac…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Asynchronous stochastic approximations with asymptotically biased errors and deep multi-agent learning

2018-02-22 · Arunselvan Ramaswamy, Shalabh Bhatnagar, Daniel E. Quevedo

Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools and techniques that are popular in multi-agent and distributed control scenarios. To counter Bellman's curse of dimensio…

Multi-agent Reinforcement Learning Policy Gradient Methods