Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing

2024-02-03 · Haobin Jiang, Ziluo Ding, Zongqing Lu

Exploration in decentralized cooperative multi-agent reinforcement learning faces two challenges. One is that the novelty of global states is unavailable, while the novelty of local observations is biased. The other is how agents can explore in a coordinated way. To address these challenges, we propose MACE, a simple yet effective multi-agent coordinated exploration method. By communicating only local novelty, agents can take into account other agents' local novelty to approximate the global novelty. Further, we newly introduce weighted mutual information to measure the influence of one agent's action on other agents' accumulated novelty. We convert it as an intrinsic reward in hindsight to encourage agents to exert more influence on other agents' exploration and boost coordinated exploration. Empirically, we show that MACE achieves superior performance in three multi-agent environments with sparse rewards.

📄 PDF Abstract BibTeX arXiv:2402.02097

Code (1)

sigmabm/mace 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning

Boltzmann-based Exploration for Robust Decentralized Multi-Agent Planning (Extended Version)

2026-03-02 · Nhat D. A. Nguyen, Duong D. Nguyen, Gianluca Rizzo, Hung X. Nguyen arxiv

Decentralized Monte Carlo Tree Search (Dec-MCTS) is widely used for cooperative multi-agent planning but struggles in sparse or skewed reward environments. We introduce Coordinated Boltzmann MCTS (CB-MCTS), which replace…

A New Framework for Multi-Agent Reinforcement Learning -- Centralized Training and Exploration with Decentralized Execution via Policy Distillation

2019-10-21 · Gang Chen

Deep reinforcement learning (DRL) is a booming area of artificial intelligence. Many practical applications of DRL naturally involve more than one collaborative learners, making it important to study DRL in a multi-agent…

Deep Reinforcement Learning Multi-agent Reinforcement Learning Reinforcement Learning

Multi-Agent Guided Policy Optimization

2025-07-24 · Yueheng Li, Guangming Xie, Zongqing Lu arxiv

Due to practical constraints such as partial observability and limited communication, Centralized Training with Decentralized Execution (CTDE) has become the dominant paradigm in cooperative Multi-Agent Reinforcement Lea…

Multi-agent Reinforcement Learning

Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Training

2021-07-29 · Piyush K. Sharma, Rolando Fernandez, Erin Zaroukian, Michael Dorothy 외

Much work has been dedicated to the exploration of Multi-Agent Reinforcement Learning (MARL) paradigms implementing a centralized learning with decentralized execution (CLDE) approach to achieve human-like collaboration …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Integrating Transit Signal Priority into Multi-Agent Reinforcement Learning based Traffic Signal Control

2024-11-28 · Dickness Kakitahi Kwesiga, Suyash Chandra Vishnoi, Angshuman Guin, Michael Hunter

This study integrates Transit Signal Priority (TSP) into multi-agent reinforcement learning (MARL) based traffic signal control. The first part of the study develops adaptive signal control based on MARL for a pair of co…

Multi-agent Reinforcement Learning Traffic Signal Control