Multi-Agent Reinforcement Learning with Selective State-Space Models
The Transformer model has demonstrated success across a wide range of domains, including in Multi-Agent Reinforcement Learning (MARL) where the Multi-Agent Transformer (MAT) has emerged as a leading algorithm in the field. However, a significant drawback of Transformer models is their quadratic computational complexity relative to input size, making them computationally expensive when scaling to larger inputs. This limitation restricts MAT's scalability in environments with many agents. Recently, State-Space Models (SSMs) have gained attention due to their computational efficiency, but their application in MARL remains unexplored. In this work, we investigate the use of Mamba, a recent SSM, in MARL and assess whether it can match the performance of MAT while providing significant improvements in efficiency. We introduce a modified version of MAT that incorporates standard and bi-directional Mamba blocks, as well as a novel "cross-attention" Mamba block. Extensive testing shows that our Multi-Agent Mamba (MAM) matches the performance of MAT across multiple standard multi-agent environments, while offering superior scalability to larger agent scenarios. This is significant for the MARL community, because it indicates that SSMs could replace Transformers without compromising performance, whilst also supporting more effective scaling to higher numbers of agents. Our project page is available at https://sites.google.com/view/multi-agent-mamba .
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencyMambaMulti-agent Reinforcement Learningreinforcement-learningReinforcement LearningState Space ModelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Selective Experience Sharing in Reinforcement Learning Enhances Interference Management
We propose a novel multi-agent reinforcement learning (RL) approach for inter-cell interference mitigation, in which agents selectively share their experiences with other agents. Each base station is equipped with an age…
ManagementMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training. The intuition behind t…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningSelective Reincarnation: Offline-to-Online Multi-Agent Reinforcement Learning
'Reincarnation' in reinforcement learning has been proposed as a formalisation of reusing prior computation from past experiments when training an agent in an environment. In this paper, we present a brief foray into the…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningSolving Continual Offline RL through Selective Weights Activation on Aligned Spaces
Continual offline reinforcement learning (CORL) has shown impressive ability in diffusion-based lifelong learning systems by modeling the joint distributions of trajectories. However, most research only focuses on limite…
Continual LearningLifelong learningOffline RLQuantizationTowards Fault Tolerance in Multi-Agent Reinforcement Learning
Agent faults pose a significant threat to the performance of multi-agent reinforcement learning (MARL) algorithms, introducing two key challenges. First, agents often struggle to extract critical information from the cha…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement Learning