Towards Generalizability of Multi-Agent Reinforcement Learning in Graphs with Recurrent Message Passing

2024-02-07 · Jannis Weil, Zhenghua Bao, Osama Abboud, Tobias Meuser

Graph-based environments pose unique challenges to multi-agent reinforcement learning. In decentralized approaches, agents operate within a given graph and make decisions based on partial or outdated observations. The size of the observed neighborhood limits the generalizability to different graphs and affects the reactivity of agents, the quality of the selected actions, and the communication overhead. This work focuses on generalizability and resolves the trade-off in observed neighborhood size with a continuous information flow in the whole graph. We propose a recurrent message-passing model that iterates with the environment's steps and allows nodes to create a global representation of the graph by exchanging messages with their neighbors. Agents receive the resulting learned graph observations based on their location in the graph. Our approach can be used in a decentralized manner at runtime and in combination with a reinforcement learning algorithm of choice. We evaluate our method across 1000 diverse graphs in the context of routing in communication networks and find that it enables agents to generalize and adapt to changes in the graph.

📄 PDF Abstract BibTeX arXiv:2402.05027

Code (1)

jw3il/graph-marl 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Leveraging weights signals -- Predicting and improving generalizability in reinforcement learning

2025-11-25 · Olivier Moulin, Vincent Francois-lavet, Paul Elbers, Mark Hoogendoorn arxiv

Generalizability of Reinforcement Learning (RL) agents (ability to perform on environments different from the ones they have been trained on) is a key problem as agents have the tendency to overfit to their training envi…

Reinforcement Learning

MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

2021-09-26 · Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang 외

Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffic, and the decision-making in complex mu…

Benchmarking Decision Making Multi-agent Reinforcement Learning reinforcement-learning+4

A Deep Policy Inference Q-Network for Multi-Agent Systems

2017-12-21 · Zhang-Wei Hong, Shih-Yang Su, Tzu-Yun Shann, Yi-Hsiang Chang 외

We present DPIQN, a deep policy inference Q-network that targets multi-agent systems composed of controllable agents, collaborators, and opponents that interact with each other. We focus on one challenging issue in such …

Deep Reinforcement Learning Reinforcement Learning

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

2021-10-24 · Mohammad Goudarzi, Marimuthu Palaniswami, Rajkumar Buyya

Fog/Edge computing is a novel computing paradigm supporting resource-constrained Internet of Things (IoT) devices by the placement of their tasks on the edge and/or cloud servers. Recently, several Deep Reinforcement Lea…

Deep Reinforcement Learning Edge-computing

What is Going on Inside Recurrent Meta Reinforcement Learning Agents?

2021-04-29 · Safa Alver, Doina Precup

Recurrent meta reinforcement learning (meta-RL) agents are agents that employ a recurrent neural network (RNN) for the purpose of "learning a learning algorithm". After being trained on a pre-specified task distribution,…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)