Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing
Sharing parameters in multi-agent deep reinforcement learning has played an essential role in allowing algorithms to scale to a large number of agents. Parameter sharing between agents significantly decreases the number of trainable parameters, shortening training times to tractable levels, and has been linked to more efficient learning. However, having all agents share the same parameters can also have a detrimental effect on learning. We demonstrate the impact of parameter sharing methods on training speed and converged returns, establishing that when applied indiscriminately, their effectiveness is highly dependent on the environment. We propose a novel method to automatically identify agents which may benefit from sharing parameters by partitioning them based on their abilities and goals. Our approach combines the increased sample efficiency of parameter sharing with the representational capacity of multiple independent networks to reduce training time and increase final returns.
Code (1)
Tasks
Deep Reinforcement LearningMulti-agent Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Scaling the Scaling Logic: Agentic Meta-Synthesis of Logic Reasoning
Reinforcement Learning from Verifiable Rewards (RLVR) is bottlenecked by data: existing synthesis pipelines rely on expert-written code or fixed templates, confining growth to instance-level perturbations. We shift the e…
Reinforcement LearningAgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning
Recent progress on long-horizon agentic tasks has been driven largely by scaling up individual agents through stronger models, better tools, and more effective scaffolding. In contrast, much less is understood about scal…
Reinforcement LearningSelectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training. The intuition behind t…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningBehavior Knowledge Merge in Reinforced Agentic Models
Reinforcement learning (RL) is central to post-training, particularly for agentic models that require specialized reasoning behaviors. In this setting, model merging offers a practical mechanism for integrating multiple …
Reinforcement LearningMulti-Agent Reinforcement Learning with Selective State-Space Models
The Transformer model has demonstrated success across a wide range of domains, including in Multi-Agent Reinforcement Learning (MARL) where the Multi-Agent Transformer (MAT) has emerged as a leading algorithm in the fiel…
Computational EfficiencyMambaMulti-agent Reinforcement Learningreinforcement-learning+2