Deep Reinforcement Learning for Adaptive Exploration of Unknown Environments
Performing autonomous exploration is essential for unmanned aerial vehicles (UAVs) operating in unknown environments. Often, these missions start with building a map for the environment via pure exploration and subsequently using (i.e. exploiting) the generated map for downstream navigation tasks. Accomplishing these navigation tasks in two separate steps is not always possible or even disadvantageous for UAVs deployed in outdoor and dynamically changing environments. Current exploration approaches either use a priori human-generated maps or use heuristics such as frontier-based exploration. Other approaches use learning but focus only on learning policies for specific tasks by either using sample inefficient random exploration or by making impractical assumptions about full map availability. In this paper, we develop an adaptive exploration approach to trade off between exploration and exploitation in one single step for UAVs searching for areas of interest (AoIs) in unknown environments using Deep Reinforcement Learning (DRL). The proposed approach uses a map segmentation technique to decompose the environment map into smaller, tractable maps. Then, a simple information gain function is repeatedly computed to determine the best target region to search during each iteration of the process. DDQN and A2C algorithms are extended with a stack of LSTM layers and trained to generate optimal policies for the exploration and exploitation, respectively. We tested our approach in 3 different tasks against 4 baselines. The results demonstrate that our proposed approach is capable of navigating through randomly generated environments and covering more AoI in less time steps compared to the baselines.
Code (1)
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding
Empowering safe exploration of reinforcement learning (RL) agents during training is a critical challenge towards their deployment in many real-world scenarios. When prior knowledge of the domain or task is unavailable, …
reinforcement-learningReinforcement Learning (RL)Safe ExplorationSafe Reinforcement LearningWhen to Go, and When to Explore: The Benefit of Post-Exploration in Intrinsic Motivation
Go-Explore achieved breakthrough performance on challenging reinforcement learning (RL) tasks with sparse rewards. The key insight of Go-Explore was that successful exploration requires an agent to first return to an int…
Reinforcement Learning (RL)Exploring Unknown States with Action Balance
Exploration is a key problem in reinforcement learning. Recently bonus-based methods have achieved considerable successes in environments where exploration is difficult such as Montezuma's Revenge, which assign additiona…
Atari GamesDeep Reinforcement LearningMontezuma's Revengereinforcement-learning+2A Survey of Exploration Methods in Reinforcement Learning
Exploration is an essential component of reinforcement learning algorithms, where agents need to learn how to predict and control unknown and often stochastic environments. Reinforcement learning agents depend crucially …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)SurveyMarsExplorer: Exploration of Unknown Terrains via Deep Reinforcement Learning and Procedurally Generated Environments
This paper is an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of unknown terrains. Within this scope, MarsExplorer, an openai-gym c…
Deep Reinforcement LearningOpenAI Gymreinforcement-learningReinforcement Learning (RL)