Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigation

2021-12-16 · Enrico Marchesini, Alessandro Farinelli

We study the problem of multi-robot mapless navigation in the popular Centralized Training and Decentralized Execution (CTDE) paradigm. This problem is challenging when each robot considers its path without explicitly sharing observations with other robots and can lead to non-stationary issues in Deep Reinforcement Learning (DRL). The typical CTDE algorithm factorizes the joint action-value function into individual ones, to favor cooperation and achieve decentralized execution. Such factorization involves constraints (e.g., monotonicity) that limit the emergence of novel behaviors in an individual as each agent is trained starting from a joint action-value. In contrast, we propose a novel architecture for CTDE that uses a centralized state-value network to compute a joint state-value, which is used to inject global state information in the value-based updates of the agents. Consequently, each model computes its gradient update for the weights, considering the overall state of the environment. Our idea follows the insights of Dueling Networks as a separate estimation of the joint state-value has both the advantage of improving sample efficiency, while providing each robot information whether the global state is (or is not) valuable. Experiments in a robotic navigation task with 2 4, and 8 robots, confirm the superior performance of our approach over prior CTDE methods (e.g., VDN, QMIX).

📄 PDF Abstract BibTeX arXiv:2112.09012

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

A State Representation Dueling Network for Deep Reinforcement Learning

2020-12-24 · Haomin Qiu, Feng Liu

In recent years there have been many successes in boosting the performance of Deep Q-Networks (DQN). Dueling DQN uses simple dueling architecture but significantly improves the performance of DQN [1]. However, Dueling DQ…

Deep Reinforcement Learning General Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reducing Dueling Bandits to Cardinal Bandits

2014-05-14 · Nir Ailon, Thorsten Joachims, Zohar Karnin

We present algorithms for reducing the Dueling Bandits problem to the conventional (stochastic) Multi-Armed Bandits problem. The Dueling Bandits problem is an online model of learning with ordinal feedback of the form "A…

Multi-Armed Bandits

Robot Navigation in a Crowd by Integrating Deep Reinforcement Learning and Online Planning

2021-02-26 · Zhiqian Zhou, Pengming Zhu, Zhiwen Zeng, Junhao Xiao 외

It is still an open and challenging problem for mobile robots navigating along time-efficient and collision-free paths in a crowd. The main challenge comes from the complex and sophisticated interaction mechanism, which …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Dueling Convex Optimization with General Preferences

2022-09-27 · Aadirupa Saha, Tomer Koren, Yishay Mansour

We address the problem of \emph{convex optimization with dueling feedback}, where the goal is to minimize a convex function given a weaker form of \emph{dueling} feedback. Each query consists of two points and the duelin…

Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples

2021-06-28 · Li Meng, Anis Yazidi, Morten Goodwin, Paal Engelstad

In this article, we propose a novel algorithm for deep reinforcement learning named Expert Q-learning. Expert Q-learning is inspired by Dueling Q-learning and aims at incorporating semi-supervised learning into reinforce…

Deep Reinforcement Learning Imitation Learning Q-Learning reinforcement-learning+2