Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance

2024-10-03 · Joshua McClellan, Naveed Haghani, John Winder, Furong Huang, Pratap Tokekar

Multi-Agent Reinforcement Learning (MARL) struggles with sample inefficiency and poor generalization [1]. These challenges are partially due to a lack of structure or inductive bias in the neural networks typically used in learning the policy. One such form of structure that is commonly observed in multi-agent scenarios is symmetry. The field of Geometric Deep Learning has developed Equivariant Graph Neural Networks (EGNN) that are equivariant (or symmetric) to rotations, translations, and reflections of nodes. Incorporating equivariance has been shown to improve learning efficiency and decrease error [ 2 ]. In this paper, we demonstrate that EGNNs improve the sample efficiency and generalization in MARL. However, we also show that a naive application of EGNNs to MARL results in poor early exploration due to a bias in the EGNN structure. To mitigate this bias, we present Exploration-enhanced Equivariant Graph Neural Networks or E2GN2. We compare E2GN2 to other common function approximators using common MARL benchmarks MPE and SMACv2. E2GN2 demonstrates a significant improvement in sample efficiency, greater final reward convergence, and a 2x-5x gain in over standard GNNs in our generalization tests. These results pave the way for more reliable and effective solutions in complex multi-agent systems.

📄 PDF Abstract BibTeX arXiv:2410.02581

Code (0)

등록된 구현이 없습니다.

Tasks

Inductive BiasMulti-agent Reinforcement Learning

Sample-Efficient Agnostic Boosting

2024-10-31 · Udaya Ghai, Karan Singh

The theory of boosting provides a computational framework for aggregating approximate weak learning algorithms, which perform marginally better than a random predictor, into an accurate strong learner. In the realizable …

DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning

2025-05-06 · Borui Wang, Kathleen McKeown, Rex Ying

Reinforcement learning from expert demonstrations has long remained a challenging research problem, and existing state-of-the-art methods using behavioral cloning plus further RL training often suffer from poor generaliz…

reinforcement-learning Reinforcement Learning

Condensed Gradient Boosting

2022-11-26 · Seyedsaman Emami, Gonzalo Martínez-Muñoz

This paper presents a computationally efficient variant of gradient boosting for multi-class classification and multi-output regression tasks. Standard gradient boosting uses a 1-vs-all strategy for classifications tasks…

Computational Efficiency Multi-class Classification regression

SigmaRL: A Sample-Efficient and Generalizable Multi-Agent Reinforcement Learning Framework for Motion Planning

2024-08-14 · Jianye Xu, Pan Hu, Bassam Alrifaee

This paper introduces an open-source, decentralized framework named SigmaRL, designed to enhance both sample efficiency and generalization of multi-agent Reinforcement Learning (RL) for motion planning of connected and a…

CPU Motion Planning Multi-agent Reinforcement Learning Reinforcement Learning (RL)

NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations

2019-06-17 · Qiaoyun Wu, Dinesh Manocha, Jun Wang, Kai Xu

We propose improving the cross-target and cross-scene generalization of visual navigation through learning an agent that is guided by conceiving the next observations it expects to see. This is achieved by learning a var…

Visual Navigation