Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments

2018-02-23 · Yan Zheng, Jianye Hao, Zongzhang Zhang

Recently, multiagent deep reinforcement learning (DRL) has received increasingly wide attention. Existing multiagent DRL algorithms are inefficient when facing with the non-stationarity due to agents update their policies simultaneously in stochastic cooperative environments. This paper extends the recently proposed weighted double estimator to the multiagent domain and propose a multiagent DRL framework, named weighted double deep Q-network (WDDQN). By utilizing the weighted double estimator and the deep neural network, WDDQN can not only reduce the bias effectively but also be extended to scenarios with raw visual inputs. To achieve efficient cooperation in the multiagent domain, we introduce the lenient reward network and the scheduled replay strategy. Experiments show that the WDDQN outperforms the existing DRL and multiaent DRL algorithms, i.e., double DQN and lenient Q-learning, in terms of the average reward and the convergence rate in stochastic cooperative environments.

📄 PDF Abstract BibTeX arXiv:1802.08534

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Double Q-learning Double Q-learning is an off-policy reinforcement learning algorithm that utilises double estimation to counteract overestimation problems with traditional Q-learning. The…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Double DQN 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem

2021-04-22 · Chengwei Zhang, Shan Jin, Wanli Xue, Xiaofei Xie 외

The adaptive traffic signal control (ATSC) problem can be modeled as a multiagent cooperative game among urban intersections, where intersections cooperate to optimize their common goal. Recently, reinforcement learning …

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

Optimal Decision-Making in Mixed-Agent Partially Observable Stochastic Environments via Reinforcement Learning

2019-01-04 · Roi Ceren

Optimal decision making with limited or no information in stochastic environments where multiple agents interact is a challenging topic in the realm of artificial intelligence. Reinforcement learning (RL) is a popular ap…

Decision Making Image Segmentation Model-based Reinforcement Learning Q-Learning+4

Safe Multiagent Coordination via Entropic Exploration

2024-12-29 · Ayhan Alp Aydeniz, Enrico Marchesini, Robert Loftin, Christopher Amato 외

Many real-world multiagent learning problems involve safety concerns. In these setups, typical safe reinforcement learning algorithms constrain agents' behavior, limiting exploration -- a crucial component for discoverin…

reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

Multiagent Soft Q-Learning

2018-04-25 · Ermo Wei, Drew Wicke, David Freelan, Sean Luke

Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic p…

Policy Gradient Methods Q-Learning reinforcement-learning Reinforcement Learning+1

Learning to Teach in Cooperative Multiagent Reinforcement Learning

2018-05-20 · Shayegan Omidshafiei, Dong-Ki Kim, Miao Liu, Gerald Tesauro 외

Collective human knowledge has clearly benefited from the fact that innovations by individuals are taught to others through communication. Similar to human social groups, agents in distributed learning systems would like…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)