홈 › Papers

Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning

2022-10-12 · Pedro P. Santos, Diogo S. Carvalho, Miguel Vasco, Alberto Sardinha, Pedro A. Santos, Ana Paiva, Francisco S. Melo

We introduce hybrid execution in multi-agent reinforcement learning (MARL), a new paradigm in which agents aim to successfully complete cooperative tasks with arbitrary communication levels at execution time by taking advantage of information-sharing among the agents. Under hybrid execution, the communication level can range from a setting in which no communication is allowed between agents (fully decentralized), to a setting featuring full communication (fully centralized), but the agents do not know beforehand which communication level they will encounter at execution time. To formalize our setting, we define a new class of multi-agent partially observable Markov decision processes (POMDPs) that we name hybrid-POMDPs, which explicitly model a communication process between the agents. We contribute MARO, an approach that makes use of an auto-regressive predictive model, trained in a centralized manner, to estimate missing agents' observations at execution time. We evaluate MARO on standard scenarios and extensions of previous benchmarks tailored to emphasize the negative impact of partial observability in MARL. Experimental results show that our method consistently outperforms relevant baselines, allowing agents to act with faulty communication while successfully exploiting shared information.

📄 PDF Abstract BibTeX arXiv:2210.06274

Code (1)

PPSantos/hybrid-marl 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

2019-03-12 · Haotian Fu, Hongyao Tang, Jianye Hao, Zihan Lei 외

Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previo…

Deep Reinforcement Learning Multi-agent Reinforcement Learning Q-Learning reinforcement-learning+2

A further exploration of deep Multi-Agent Reinforcement Learning with Hybrid Action Space

2022-08-30 · Hongzhi Hua, Guixuan Wen, Kaigui Wu

The research of extending deep reinforcement learning (drl) to multi-agent field has solved many complicated problems and made great achievements. However, almost all these studies only focus on discrete or continuous ac…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy

2022-06-10 · Hongzhi Hua, Kaigui Wu, Guixuan Wen

Multi-agent deep reinforcement learning has been applied to address a variety of complex problems with either discrete or continuous action spaces and achieved great success. However, most real-world environments cannot …

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization

2022-09-26 · Jiangxing Wang, Deheng Ye, Zongqing Lu

In cooperative multi-agent reinforcement learning (MARL), combining value decomposition with actor-critic enables agents to learn stochastic policies, which are more suitable for the partially observable environment. Giv…

Multi-agent Reinforcement Learning

Decentralized Multi-Agent Actor-Critic with Generative Inference

2019-10-07 · Kevin Corder, Manuel M. Vindiola, Keith Decker

Recent multi-agent actor-critic methods have utilized centralized training with decentralized execution to address the non-stationarity of co-adapting agents. This training paradigm constrains learning to the centralized…