Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning

2024-09-01 · Jiaming Yin, Weixiong Rao, Yu Xiao, Keshuang Tang

In this paper, we study the shortest path problem (SPP) with multiple source-destination pairs (MSD), namely MSD-SPP, to minimize average travel time of all shortest paths. The inherent traffic capacity limits within a road network contributes to the competition among vehicles. Multi-agent reinforcement learning (MARL) model cannot offer effective and efficient path planning cooperation due to the asynchronous decision making setting in MSD-SPP, where vehicles (a.k.a agents) cannot simultaneously complete routing actions in the previous time step. To tackle the efficiency issue, we propose to divide an entire road network into multiple sub-graphs and subsequently execute a two-stage process of inter-region and intra-region route planning. To address the asynchronous issue, in the proposed asyn-MARL framework, we first design a global state, which exploits a low-dimensional vector to implicitly represent the joint observations and actions of multi-agents. Then we develop a novel trajectory collection mechanism to decrease the redundancy in training trajectories. Additionally, we design a novel actor network to facilitate the cooperation among vehicles towards the same or close destinations and a reachability graph aimed at preventing infinite loops in routing paths. On both synthetic and real road networks, our evaluation result demonstrates that our approach outperforms state-of-the-art planning approaches.

📄 PDF Abstract BibTeX arXiv:2409.00754

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Travel 설명 없음

Multiagent Soft Q-Learning

2018-04-25 · Ermo Wei, Drew Wicke, David Freelan, Sean Luke

Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic p…

Policy Gradient Methods Q-Learning reinforcement-learning Reinforcement Learning+1

Agent Modeling as Auxiliary Task for Deep Reinforcement Learning

2019-07-22 · Pablo Hernandez-Leal, Bilal Kartal, Matthew E. Taylor

In this paper we explore how actor-critic methods in deep reinforcement learning, in particular Asynchronous Advantage Actor-Critic (A3C), can be extended with agent modeling. Inspired by recent works on representation l…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments

2018-02-23 · Yan Zheng, Jianye Hao, Zongzhang Zhang

Recently, multiagent deep reinforcement learning (DRL) has received increasingly wide attention. Existing multiagent DRL algorithms are inefficient when facing with the non-stationarity due to agents update their policie…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Individual Planning in Infinite-Horizon Multiagent Settings: Inference, Structure and Scalability

2015-12-01 · NeurIPS 2015 12 · Xia Qu, Prashant Doshi

This paper provides the first formalization of self-interested planning in multiagent settings using expectation-maximization (EM). Our formalization in the context of infinite-horizon and finitely-nest…

Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots

2024-10-08 · Milad Farjadnasab, Shahin Sirouspour

Cooperative mission planning for heterogeneous teams of mobile robots presents a unique set of challenges, particularly when operating under communication constraints and limited computational resources. To address these…

Decision Making Multi-agent Reinforcement Learning