On-policy Actor-Critic Reinforcement Learning for Multi-UAV Exploration

2024-09-17 · Ali Moltajaei Farid, Jafar Roshanian, Malek Mouhoub

Unmanned aerial vehicles (UAVs) have become increasingly popular in various fields, including precision agriculture, search and rescue, and remote sensing. However, exploring unknown environments remains a significant challenge. This study aims to address this challenge by utilizing on-policy Reinforcement Learning (RL) with Proximal Policy Optimization (PPO) to explore the {two dimensional} area of interest with multiple UAVs. The UAVs will avoid collision with obstacles and each other and do the exploration in a distributed manner. The proposed solution includes actor-critic networks using deep convolutional neural networks {(CNN)} and long short-term memory (LSTM) for identifying the UAVs and areas that have already been covered. Compared to other RL techniques, such as policy gradient (PG) and asynchronous advantage actor-critic (A3C), the simulation results demonstrate the superiority of the proposed PPO approach. Also, the results show that combining LSTM with CNN in critic can improve exploration. Since the proposed exploration has to work in unknown environments, the results showed that the proposed setup can complete the coverage when we have new maps that differ from the trained maps. Finally, we showed how tuning hyper parameters may affect the overall performance.

📄 PDF Abstract BibTeX arXiv:2409.11058

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

Tanh Activation 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

Sigmoid Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Wasserstein Barycenter Soft Actor-Critic

2025-06-11 · Zahra Shahrooei, Ali Baheri

Deep off-policy actor-critic algorithms have emerged as the leading framework for reinforcement learning in continuous control domains. However, most of these algorithms suffer from poor sample efficiency, especially in …

continuous-control Continuous Control MuJoCo

Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics

2019-03-11 · Denis Steckelmacher, Hélène Plisnier, Diederik M. Roijers, Ann Nowé

Value-based reinforcement-learning algorithms provide state-of-the-art results in model-free discrete-action settings, and tend to outperform actor-critic algorithms. We argue that actor-critic algorithms are limited by …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

Learning to Explore with Meta-Policy Gradient

2018-03-13 · Tianbing Xu, Qiang Liu, Liang Zhao, Jian Peng

The performance of off-policy learning, including deep Q-learning and deep deterministic policy gradient (DDPG), critically depends on the choice of the exploration policy. Existing exploration methods are mostly based o…

Q-Learning Reinforcement Learning Reinforcement Learning (RL)

Learning to Explore via Meta-Policy Gradient

2018-07-01 · ICML 2018 7 · Tianbing Xu, Qiang Liu, Liang Zhao, Jian Peng

continuous-control Continuous Control Q-Learning reinforcement-learning+2

Virtual Action Actor-Critic Framework for Exploration (Student Abstract)

2023-11-06 · Bumgeun Park, TaeYoung Kim, Quoc-Vinh Lai-Dang, Dongsoo Har

Efficient exploration for an agent is challenging in reinforcement learning (RL). In this paper, a novel actor-critic framework namely virtual action actor-critic (VAAC), is proposed to address the challenge of efficient…

Efficient Exploration Reinforcement Learning (RL)