Joint Intrinsic Motivation for Coordinated Exploration in Multi-Agent Deep Reinforcement Learning

2024-02-06 · Maxime Toquebiau, Nicolas Bredeche, Faïz Benamar, Jae-Yun Jun

Multi-agent deep reinforcement learning (MADRL) problems often encounter the challenge of sparse rewards. This challenge becomes even more pronounced when coordination among agents is necessary. As performance depends not only on one agent's behavior but rather on the joint behavior of multiple agents, finding an adequate solution becomes significantly harder. In this context, a group of agents can benefit from actively exploring different joint strategies in order to determine the most efficient one. In this paper, we propose an approach for rewarding strategies where agents collectively exhibit novel behaviors. We present JIM (Joint Intrinsic Motivation), a multi-agent intrinsic motivation method that follows the centralized learning with decentralized execution paradigm. JIM rewards joint trajectories based on a centralized measure of novelty designed to function in continuous environments. We demonstrate the strengths of this approach both in a synthetic environment designed to reveal shortcomings of state-of-the-art MADRL methods, and in simulated robotic tasks. Results show that joint exploration is crucial for solving tasks where the optimal strategy requires a high level of coordination.

📄 PDF Abstract BibTeX arXiv:2402.03972

Code (1)

mtoquebiau/joint-intrinsic-motivation 공식 구현 pytorch

Tasks

Deep Reinforcement Learning

Intrinsic-Motivation Multi-Robot Social Formation Navigation with Coordinated Exploration

2025-12-15 · Hao Fu, Wei Liu, Shuai Zhou arxiv

This paper investigates the application of reinforcement learning (RL) to multi-robot social formation navigation, a critical capability for enabling seamless human-robot coexistence. While RL offers a promising paradigm…

Reinforcement Learning

Influence-Based Reinforcement Learning for Intrinsically-Motivated Agents

2021-08-28 · Ammar Fayad, Majd Ibrahim

Discovering successful coordinated behaviors is a central challenge in Multi-Agent Reinforcement Learning (MARL) since it requires exploring a joint action space that grows exponentially with the number of agents. In thi…

counterfactual Multi-agent Reinforcement Learning OpenAI Gym reinforcement-learning+3

AdaMemento: Adaptive Memory-Assisted Policy Optimization for Reinforcement Learning

2024-10-06 · Renye Yan, Yaozhong Gan, You Wu, Junliang Xing 외

In sparse reward scenarios of reinforcement learning (RL), the memory mechanism provides promising shortcuts to policy optimization by reflecting on past experiences like humans. However, current memory-based RL methods …

Ensemble Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Efficient Exploration through Intrinsic Motivation Learning for Unsupervised Subgoal Discovery in Model-Free Hierarchical Reinforcement Learning

2019-11-18 · Jacob Rafati, David C. Noelle

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of explorati…

Efficient Exploration Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Intrinsic Motivation for Encouraging Synergistic Behavior

2020-02-12 · ICLR 2020 1 · Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, Abhinav Gupta

We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks, which are tasks where multiple agents must work together to achieve a goal they could not in…