Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms
We benchmark commonly used multi-agent deep reinforcement learning (MARL) algorithms on a variety of cooperative multi-agent games. While there has been significant innovation in MARL algorithms, algorithms tend to be tested and tuned on a single domain and their average performance across multiple domains is less characterized. Furthermore, since the hyperparameters of the algorithms are carefully tuned to the task of interest, it is unclear whether hyperparameters can easily be found that allow the algorithm to be repurposed for other cooperative tasks with different reward structure and environment dynamics. To investigate the consistency of the performance of MARL algorithms, we build an open-source library of multi-agent algorithms including DDPG/TD3/SAC with centralized Q functions, PPO with centralized value functions, and QMix and test them across a range of tasks that vary in coordination difficulty and agent number. The domains include the particle-world environments, starcraft micromanagement challenges, the Hanabi challenge, and the hide-and-seek environments. Finally, we investigate the ease of hyper-parameter tuning for each of the algorithms by tuning hyper-parameters in one environment per domain and re-using them in the other environments within the domain.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)StarcraftMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment
Deep reinforcement learning is actively used for training autonomous car policies in a simulated driving environment. Due to the large availability of various reinforcement learning algorithms and the lack of their syste…
Autonomous DrivingBenchmarkingDeep Reinforcement Learningreinforcement-learning+2BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
The field of Multi-Agent Reinforcement Learning (MARL) is currently facing a reproducibility crisis. While solutions for standardized reporting have been proposed to address the issue, we still lack a benchmarking tool t…
BenchmarkingMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks
Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and com…
BenchmarkingDeep Reinforcement Learningreinforcement-learningReinforcement Learning+1SustainDC: Benchmarking for Sustainable Data Center Control
Machine learning has driven an exponential increase in computational demand, leading to massive data centers that consume significant amounts of energy and contribute to climate change. This makes sustainable data center…
BenchmarkingManagementMulti-agent Reinforcement LearningMultiobjective Optimization+1A Toolkit for Reliable Benchmarking and Research in Multi-Objective Reinforcement Learning
Multi-objective reinforcement learning algorithms (MORL) extend standard reinforcement learning (RL) to scenarios where agents must optimize multiple---potentially conflicting---objectives, each represented by a distinct…
BenchmarkingMulti-Objective Reinforcement Learningreinforcement-learningReinforcement Learning+1