MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning
Population-based multi-agent reinforcement learning (PB-MARL) refers to the series of methods nested with reinforcement learning (RL) algorithms, which produces a self-generated sequence of tasks arising from the coupled population dynamics. By leveraging auto-curricula to induce a population of distinct emergent strategies, PB-MARL has achieved impressive success in tackling multi-agent tasks. Despite remarkable prior arts of distributed RL frameworks, PB-MARL poses new challenges for parallelizing the training frameworks due to the additional complexity of multiple nested workloads between sampling, training and evaluation involved with heterogeneous policy interactions. To solve these problems, we present MALib, a scalable and efficient computing framework for PB-MARL. Our framework is comprised of three key components: (1) a centralized task dispatching model, which supports the self-generated tasks and scalable training with heterogeneous policy combinations; (2) a programming architecture named Actor-Evaluator-Learner, which achieves high parallelism for both training and sampling, and meets the evaluation requirement of auto-curriculum learning; (3) a higher-level abstraction of MARL training paradigms, which enables efficient code reuse and flexible deployments on different distributed computing paradigms. Experiments on a series of complex tasks such as multi-agent Atari Games show that MALib achieves throughput higher than 40K FPS on a single machine with $32$ CPU cores; 5x speedup than RLlib and at least 3x speedup than OpenSpiel in multi-agent training tasks. MALib is publicly available at https://github.com/sjtu-marl/malib.
Code (1)
Tasks
Atari GamesCPUDistributed ComputingMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)rllibSimilar Papers 제목 키워드 기반
An Empirical Study on Google Research Football Multi-agent Scenarios
Few multi-agent reinforcement learning (MARL) research on Google Research Football (GRF) focus on the 11v11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been rel…
BenchmarkingMulti-agent Reinforcement Learningreinforcement-learningReinforcement LearningMALIBU Benchmark: Multi-Agent LLM Implicit Bias Uncovered
Multi-agent systems, which consist of multiple AI models interacting within a shared environment, are increasingly used for persona-based interactions. However, if not carefully designed, these systems can reinforce impl…
FairnessAnomalib: A Deep Learning Library for Anomaly Detection
This paper introduces anomalib, a novel library for unsupervised anomaly detection and localization. With reproducibility and modularity in mind, this open-source library provides algorithms from the literature and a set…
Anomaly DetectionDeep LearningModel OptimizationUnsupervised Anomaly DetectionEvolution of Societies via Reinforcement Learning
The universe involves many independent co-learning agents as an ever-evolving part of our observed environment. Yet, in practice, Multi-Agent Reinforcement Learning (MARL) applications are typically constrained to small,…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningPNS: Population-Guided Novelty Search for Reinforcement Learning in Hard Exploration Environments
Reinforcement Learning (RL) has made remarkable achievements, but it still suffers from inadequate exploration strategies, sparse reward signals, and deceptive reward functions. To alleviate these problems, a Population-…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+1