BenchMARL: Benchmarking Multi-Agent Reinforcement Learning

2023-12-03 · Matteo Bettini, Amanda Prorok, Vincent Moens

The field of Multi-Agent Reinforcement Learning (MARL) is currently facing a reproducibility crisis. While solutions for standardized reporting have been proposed to address the issue, we still lack a benchmarking tool that enables standardization and reproducibility, while leveraging cutting-edge Reinforcement Learning (RL) implementations. In this paper, we introduce BenchMARL, the first MARL training library created to enable standardized benchmarking across different algorithms, models, and environments. BenchMARL uses TorchRL as its backend, granting it high performance and maintained state-of-the-art implementations while addressing the broad community of MARL PyTorch users. Its design enables systematic configuration and reporting, thus allowing users to create and run complex benchmarks from simple one-line inputs. BenchMARL is open-sourced on GitHub: https://github.com/facebookresearch/BenchMARL

📄 PDF Abstract BibTeX arXiv:2312.01472

Code (1)

facebookresearch/benchmarl 공식 구현 pytorch

Tasks

Benchmarking Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Library 설명 없음

Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment

2021-12-22 · Aizaz Sharif, Dusica Marijan

Deep reinforcement learning is actively used for training autonomous car policies in a simulated driving environment. Due to the large availability of various reinforcement learning algorithms and the lack of their syste…

Autonomous Driving Benchmarking Deep Reinforcement Learning reinforcement-learning+2

Simulation-Based Benchmarking of Reinforcement Learning Agents for Personalized Retail Promotions

2024-05-16 · Yu Xia, Sriram Narayanamoorthy, Zhengyuan Zhou, Joshua Mabry

The development of open benchmarking platforms could greatly accelerate the adoption of AI agents in retail. This paper presents comprehensive simulations of customer shopping behaviors for the purpose of benchmarking re…

Benchmarking Reinforcement Learning (RL)

Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation

2026-05-26 · Heng Qu, Yike Liu, Renren Jin, Wenzong Zhang 외 arxiv

Vision-Language Models (VLMs) have shown rapid progress in mobile GUI navigation. This paper presents a systematic study of data scaling, benchmarking, and reasoning for VLM-based agents in this domain. To facilitate rig…

Reinforcement Learning

MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning

2024-07-23 · Florian Felten, Umut Ucak, Hicham Azmani, Gao Peng 외

Many challenging tasks such as managing traffic systems, electricity grids, or supply chains involve complex decision-making processes that must balance multiple conflicting objectives and coordinate the actions of vario…

Benchmarking Decision Making Multi-agent Reinforcement Learning Multi-Objective Multi-Agent Reinforcement Learning+3

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

2020-06-14 · Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, Stefano V. Albrecht

Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and com…

Benchmarking Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1