FoX: Formation-aware exploration in multi-agent reinforcement learning

2023-08-22 · Yonghyeon Jo, Sunwoo Lee, Junghyuk Yeom, Seungyul Han

Recently, deep multi-agent reinforcement learning (MARL) has gained significant popularity due to its success in various cooperative multi-agent tasks. However, exploration still remains a challenging problem in MARL due to the partial observability of the agents and the exploration space that can grow exponentially as the number of agents increases. Firstly, in order to address the scalability issue of the exploration space, we define a formation-based equivalence relation on the exploration space and aim to reduce the search space by exploring only meaningful states in different formations. Then, we propose a novel formation-aware exploration (FoX) framework that encourages partially observable agents to visit the states in diverse formations by guiding them to be well aware of their current formation solely based on their own observations. Numerical results show that the proposed FoX framework significantly outperforms the state-of-the-art MARL algorithms on Google Research Football (GRF) and sparse Starcraft II multi-agent challenge (SMAC) tasks.

📄 PDF Abstract BibTeX arXiv:2308.11272

Code (1)

hyeon1996/fox 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning SMAC SMAC+StarcraftStarcraft II

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Successor-Predecessor Intrinsic Exploration

2023-05-24 · NeurIPS 2023 11 · Changmin Yu, Neil Burgess, Maneesh Sahani, Samuel J. Gershman

Exploration is essential in reinforcement learning, particularly in environments where external rewards are sparse. Here we focus on exploration with intrinsic rewards, where the agent transiently augments the external r…

Atari Games Deep Reinforcement Learning Efficient Exploration reinforcement-learning+1

Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization

2026-05-09 · Xingyuan Hua, Sheng Yue, Ju Ren arxiv

Recent advancements in agentic test-time scaling allow models to gather environmental feedback before committing to final actions. A key limitation of existing methods is that they typically employ undifferentiated explo…

Reinforcement Learning

Contingency-Aware Exploration in Reinforcement Learning

2018-11-05 · ICLR 2019 5 · Jongwook Choi, Yijie Guo, Marcin Moczulski, Junhyuk Oh 외

This paper investigates whether learning contingency-awareness and controllable aspects of an environment can lead to better exploration in reinforcement learning. To investigate this question, we consider an instantiati…

Atari Games Montezuma's Revenge reinforcement-learning Reinforcement Learning+1

Cost-Aware Diffusion Active Search

2026-02-23 · Arundhati Banerjee, Jeff Schneider arxiv

Active search for recovering objects of interest through online, adaptive decision making with autonomous agents requires trading off exploration of unknown environments with exploitation of prior observations in the sea…

Reinforcement Learning Decision Making

Influence-Based Multi-Agent Exploration

2019-10-12 · ICLR 2020 1 · Tonghan Wang, Jianhao Wang, Yi Wu, Chongjie Zhang

Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent f…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)