Optimistic ε-Greedy Exploration for Cooperative Multi-Agent Reinforcement Learning

2025-02-05 · Ruoning Zhang, Siying Wang, Wenyu Chen, Yang Zhou, Zhitong Zhao, Zixuan Zhang, Ruijie Zhang

The Centralized Training with Decentralized Execution (CTDE) paradigm is widely used in cooperative multi-agent reinforcement learning. However, due to the representational limitations of traditional monotonic value decomposition methods, algorithms can underestimate optimal actions, leading policies to suboptimal solutions. To address this challenge, we propose Optimistic $\epsilon$-Greedy Exploration, focusing on enhancing exploration to correct value estimations. The underestimation arises from insufficient sampling of optimal actions during exploration, as our analysis indicated. We introduce an optimistic updating network to identify optimal actions and sample actions from its distribution with a probability of $\epsilon$ during exploration, increasing the selection frequency of optimal actions. Experimental results in various environments reveal that the Optimistic $\epsilon$-Greedy Exploration effectively prevents the algorithm from suboptimal solutions and significantly improves its performance compared to other algorithms.

📄 PDF Abstract BibTeX arXiv:2502.03506

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning

Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learning

2023-03-03 · Jihwan Oh, Joonkee Kim, Minchan Jeong, Se-Young Yun

The multi-agent setting is intricate and unpredictable since the behaviors of multiple agents influence one another. To address this environmental uncertainty, distributional reinforcement learning algorithms that incorp…

Distributional Reinforcement Learning Multi-agent Reinforcement Learning quantile regression reinforcement-learning+2

Conditionally Optimistic Exploration for Cooperative Deep Multi-Agent Reinforcement Learning

2023-03-16 · Xutong Zhao, Yangchen Pan, Chenjun Xiao, Sarath Chandar 외

Efficient exploration is critical in cooperative deep Multi-Agent Reinforcement Learning (MARL). In this work, we propose an exploration method that effectively encourages cooperative exploration based on the idea of seq…

Efficient Exploration Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Strategically Efficient Exploration in Competitive Multi-agent Reinforcement Learning

2021-07-30 · Robert Loftin, Aadirupa Saha, Sam Devlin, Katja Hofmann

High sample complexity remains a barrier to the application of reinforcement learning (RL), particularly in multi-agent systems. A large body of work has demonstrated that exploration mechanisms based on the principle of…

Efficient Exploration Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

On Optimistic versus Randomized Exploration in Reinforcement Learning

2017-06-13 · Ian Osband, Benjamin Van Roy

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each s…

Computational Efficiency reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning

2020-06-15 · NeurIPS 2020 12 · Sebastian Curi, Felix Berkenkamp, Andreas Krause

Model-based reinforcement learning algorithms with probabilistic dynamical models are amongst the most data-efficient learning methods. This is often attributed to their ability to distinguish between epistemic and aleat…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)