Do Deep Reinforcement Learning Algorithms really Learn to Navigate?
Deep reinforcement learning (DRL) algorithms have demonstrated progress in learning to find a goal in challenging environments. As the title of the paper by Mirowski et al. (2016) suggests, one might assume that DRL-based algorithms are able to “learn to navigate” and are thus ready to replace classical mapping and path-planning algorithms, at least in simulated environments. Yet, from experiments and analysis in this earlier work, it is not clear what strategies are used by these algorithms in navigating the mazes and finding the goal. In this paper, we pose and study this underlying question: are DRL algorithms doing some form of mapping and/or path-planning? Our experiments show that the algorithms are not memorizing the maps of mazes at the testing stage but, rather, at the training stage. Hence, the DRL algorithms fall short of qualifying as mapping or path-planning algorithms with any reasonable definition of mapping. We extend the experiments in Mirowski et al. (2016) by separating the set of training and testing maps and by a more ablative coverage of the space of experiments. Our systematic experiments show that the NavA3C-D1-D2-L algorithm, when trained and tested on the same maps, is able to choose the shorter paths to the goal. However, when tested on unseen maps the algorithm utilizes a wall-following strategy to find the goal without doing any mapping or path planning.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningNavigatereinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
No RL, No Simulation: Learning to Navigate without Navigating
Most prior methods for learning navigation policies require access to simulation environments, as they need online policy interaction and rely on ground-truth maps for rewards. However, building simulators is expensive (…
NavigateReinforcement Learning (RL)Performance Comparisons of Reinforcement Learning Algorithms for Sequential Experimental Design
Recent developments in sequential experimental design look to construct a policy that can efficiently navigate the design space, in a way that maximises the expected information gain. Whilst there is work on achieving tr…
Experimental DesignNavigatereinforcement-learningReinforcement LearningOpenTensor: Reproducing Faster Matrix Multiplication Discovering Algorithms
OpenTensor is a reproduction of AlphaTensor, which discovered a new algorithm that outperforms the state-of-the-art methods for matrix multiplication by Deep Reinforcement Learning (DRL). While AlphaTensor provides a pro…
Deep Reinforcement Learningreinforcement-learningMIRACLE: Inverse Reinforcement and Curriculum Learning Model for Human-inspired Mobile Robot Navigation
In emergency scenarios, mobile robots must navigate like humans, interpreting stimuli to locate potential victims rapidly without interfering with first responders. Existing socially-aware navigation algorithms face comp…
NavigateRobot NavigationAutonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Review
There is an increasing demand for using Unmanned Aerial Vehicle (UAV), known as drones, in different applications such as packages delivery, traffic monitoring, search and rescue operations, and military combat engagemen…
Navigatereinforcement-learningReinforcement Learning (RL)