Influence-Based Reinforcement Learning for Intrinsically-Motivated Agents

2021-08-28 · Ammar Fayad, Majd Ibrahim

Discovering successful coordinated behaviors is a central challenge in Multi-Agent Reinforcement Learning (MARL) since it requires exploring a joint action space that grows exponentially with the number of agents. In this paper, we propose a mechanism for achieving sufficient exploration and coordination in a team of agents. Specifically, agents are rewarded for contributing to a more diversified team behavior by employing proper intrinsic motivation functions. To learn meaningful coordination protocols, we structure agents' interactions by introducing a novel framework, where at each timestep, an agent simulates counterfactual rollouts of its policy and, through a sequence of computations, assesses the gap between other agents' current behaviors and their targets. Actions that minimize the gap are considered highly influential and are rewarded. We evaluate our approach on a set of challenging tasks with sparse rewards and partial observability that require learning complex cooperative strategies under a proper exploration scheme, such as the StarCraft Multi-Agent Challenge. Our methods show significantly improved performances over different baselines across all tasks.

📄 PDF Abstract BibTeX arXiv:2108.12581

Code (0)

등록된 구현이 없습니다.

Tasks

counterfactualMulti-agent Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Starcraft

Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

2020-12-17 · Cédric Colas, Tristan Karch, Olivier Sigaud, Pierre-Yves Oudeyer

Building autonomous machines that can explore open-ended environments, discover possible interactions and build repertoires of skills is a general objective of artificial intelligence. Developmental approaches argue that…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Influence-Based Multi-Agent Exploration

2019-10-12 · ICLR 2020 1 · Tonghan Wang, Jianhao Wang, Yi Wu, Chongjie Zhang

Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent f…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents

2024-06-03 · John L. Zhou, Weizhe Hong, Jonathan C. Kao

Cooperation between self-interested individuals is a widespread phenomenon in the natural world, but remains elusive in interactions between artificially intelligent agents. Instead, naive reinforcement learning algorith…

reinforcement-learning Reinforcement Learning

RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning

2024-05-29 · Mingqi Yuan, Roger Creus Castanyer, Bo Li, Xin Jin 외

Extrinsic rewards can effectively guide reinforcement learning (RL) agents in specific tasks. However, extrinsic rewards frequently fall short in complex environments due to the significant human effort needed for their …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

2018-10-15 · Cédric Colas, Pierre Fournier, Olivier Sigaud, Mohamed Chetouani 외

In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivated exploration. They may consider a large diversity of goals, aiming to disco…

Diversity Efficient Exploration Multi-Goal Reinforcement Learning reinforcement-learning+2