Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning

2022-09-20 · Yuchen Xiao, Weihao Tan, Christopher Amato

Synchronizing decisions across multiple agents in realistic settings is problematic since it requires agents to wait for other agents to terminate and communicate about termination reliably. Ideally, agents should learn and execute asynchronously instead. Such asynchronous methods also allow temporally extended actions that can take different amounts of time based on the situation and action executed. Unfortunately, current policy gradient methods are not applicable in asynchronous settings, as they assume that agents synchronously reason about action selection at every time step. To allow asynchronous learning and decision-making, we formulate a set of asynchronous multi-agent actor-critic methods that allow agents to directly optimize asynchronous policies in three standard training paradigms: decentralized learning, centralized learning, and centralized training for decentralized execution. Empirical results (in simulation and hardware) in a variety of realistic domains demonstrate the superiority of our approaches in large multi-agent problems and validate the effectiveness of our algorithms for learning high-quality and asynchronous solutions.

📄 PDF Abstract BibTeX arXiv:2209.10113

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Multi-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

2021-05-02 · Jiawei Wang, Lijun Sun

The bus system is a critical component of sustainable urban transportation. However, due to the significant uncertainties in passenger demand and traffic conditions, bus operation is unstable in nature and bus bunching h…

Graph Attention Inductive Learning Multi-agent Reinforcement Learning reinforcement-learning+2

Agent Modeling as Auxiliary Task for Deep Reinforcement Learning

2019-07-22 · Pablo Hernandez-Leal, Bilal Kartal, Matthew E. Taylor

In this paper we explore how actor-critic methods in deep reinforcement learning, in particular Asynchronous Advantage Actor-Critic (A3C), can be extended with agent modeling. Inspired by recent works on representation l…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems

2025-06-03 · Frederico Metelo, Alexandre Oliveira, Stevo Racković, Pedro Ákos Costa 외

Edge computing addresses the growing data demands of connected-device networks by placing computational resources closer to end users through decentralized infrastructures. This decentralization challenges traditional, f…

Edge-computing

Asynchronous Advantage Actor-Critic Agent for Starcraft II

2018-07-22 · Basel Alghanem, Keerthana P G

Deep reinforcement learning, and especially the Asynchronous Advantage Actor-Critic algorithm, has been successfully used to achieve super-human performance in a variety of video games. Starcraft II is a new challenge fo…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

Asynchronous Methods for Deep Reinforcement Learning

2016-02-04 · Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves 외

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of …

Atari Games CPU Deep Reinforcement Learning GPU+3