Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

2019-01-12 · Emanuele Pesce, Giovanni Montana

Deep reinforcement learning algorithms have recently been used to train multiple interacting agents in a centralised manner whilst keeping their execution decentralised. When the agents can only acquire partial observations and are faced with tasks requiring coordination and synchronisation skills, inter-agent communication plays an essential role. In this work, we propose a framework for multi-agent training using deep deterministic policy gradients that enables concurrent, end-to-end learning of an explicit communication protocol through a memory device. During training, the agents learn to perform read and write operations enabling them to infer a shared representation of the world. We empirically demonstrate that concurrent learning of the communication device and individual policies can improve inter-agent coordination and performance in small-scale systems. Our experimental results show that the proposed method achieves superior performance in scenarios with up to six agents. We illustrate how different communication patterns can emerge on six different tasks of increasing complexity. Furthermore, we study the effects of corrupting the communication channel, provide a visualisation of the time-varying memory content as the underlying task is being solved and validate the building blocks of the proposed memory device through ablation studies.

📄 PDF Abstract BibTeX arXiv:1901.03887

Code (1)

murtazarang/MD-MADDPG tf

Tasks

Deep Reinforcement LearningReinforcement Learning Reinforcement Learning (RL)

PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning

2022-02-20 · Zixuan Cao, Mengzhi Shi, Zhanbo Zhao, Xiujun Ma

Being difficult to scale poses great problems in multi-agent coordination. Multi-agent Reinforcement Learning (MARL) algorithms applied in small-scale multi-agent systems are hard to extend to large-scale ones because th…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning (RL)

Emergent Coordination and Phase Structure in Independent Multi-Agent Reinforcement Learning

2025-11-28 · Azusa Yamaguchi arxiv

A clearer understanding of when coordination emerges, fluctuates, or collapses in decentralized multi-agent reinforcement learning (MARL) is increasingly sought in order to characterize the dynamics of multi-agent learni…

Multi-agent Reinforcement Learning

Centralised rehearsal of decentralised cooperation: Multi-agent reinforcement learning for the scalable coordination of residential energy flexibility

2023-05-30 · Flora Charbonnier, Bei Peng, Thomas Morstyn, Malcolm McCulloch

This paper investigates how deep multi-agent reinforcement learning can enable the scalable and privacy-preserving coordination of residential energy flexibility. The coordination of distributed resources such as electri…

Multi-agent Reinforcement Learning Privacy Preserving reinforcement-learning Reinforcement Learning

A Structured Prediction Approach for Generalization in Cooperative Multi-Agent Reinforcement Learning

2019-10-19 · NeurIPS 2019 12 · Nicolas Carion, Gabriel Synnaeve, Alessandro Lazaric, Nicolas Usunier

Effective coordination is crucial to solve multi-agent collaborative (MAC) problems. While centralized reinforcement learning methods can optimally solve small MAC instances, they do not scale to large problems and they …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

Distributed Area Coverage with High Altitude Balloons Using Multi-Agent Reinforcement Learning

2025-10-04 · Adam Haroon, Tristan Schuler arxiv

High Altitude Balloons (HABs) can leverage stratospheric wind layers for limited horizontal control, enabling applications in reconnaissance, environmental monitoring, and communications networks. Existing multi-agent HA…