Harnessing Reinforcement Learning for Neural Motion Planning

2019-06-01 · Tom Jurgenson, Aviv Tamar

Motion planning is an essential component in most of today's robotic applications. In this work, we consider the learning setting, where a set of solved motion planning problems is used to improve the efficiency of motion planning on different, yet similar problems. This setting is important in applications with rapidly changing environments such as in e-commerce, among others. We investigate a general deep learning based approach, where a neural network is trained to map an image of the domain, the current robot state, and a goal robot state to the next robot state in the plan. We focus on the learning algorithm, and compare supervised learning methods with reinforcement learning (RL) algorithms. We first establish that supervised learning approaches are inferior in their accuracy due to insufficient data on the boundary of the obstacles, an issue that RL methods mitigate by actively exploring the domain. We then propose a modification of the popular DDPG RL algorithm that is tailored to motion planning domains, by exploiting the known model in the problem and the set of solved plans in the data. We show that our algorithm, dubbed DDPG-MP, significantly improves the accuracy of the learned motion planning policy. Finally, we show that given enough training data, our method can plan significantly faster on novel domains than off-the-shelf sampling based motion planners. Results of our experiments are shown in https://youtu.be/wHQ4Y4mBRb8.

📄 PDF Abstract BibTeX arXiv:1906.00214

Code (1)

tomjur/ModelBasedDDPG 공식 구현 tf

Tasks

Motion Planning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Weight Decay 설명 없음

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Adam 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Batch Normalization 설명 없음

DDPG DDPG, or Deep Deterministic Policy Gradient, is an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action…

A review of motion planning algorithms for intelligent robotics

2021-02-04 · Chengmin Zhou, Bingding Huang, Pasi Fränti

We investigate and analyze principles of typical motion planning algorithms. These include traditional planning algorithms, supervised learning, optimal value reinforcement learning, policy gradient reinforcement learnin…

Motion Planning Q-Learning reinforcement-learning Reinforcement Learning+1

Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domains

2023-02-24 · Jingwei Zhang, Jost Tobias Springenberg, Arunkumar Byravan, Leonard Hasenclever 외

In this paper we study the problem of learning multi-step dynamics prediction models (jumpy models) from unlabeled experience and their utility for fast inference of (high-level) plans in downstream tasks. In particular …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Zero-shot Generalization

Deep Reinforcement Learning for Motion Planning of Mobile Robots

2019-12-19 · Leonid Butyrev, Thorsten Edelhäußer, Christopher Mutschler

This paper presents a novel motion and trajectory planning algorithm for nonholonomic mobile robots that uses recent advances in deep reinforcement learning. Starting from a random initial state, i.e., position, velocity…

Deep Reinforcement Learning Motion Planning Position reinforcement-learning+3

Planning the path with Reinforcement Learning: Optimal Robot Motion Planning in RoboCup Small Size League Environments

2024-04-23 · Mateus G. Machado, João G. Melo, Cleber Zanchettin, Pedro H. M. Braga 외

This work investigates the potential of Reinforcement Learning (RL) to tackle robot motion planning challenges in the dynamic RoboCup Small Size League (SSL). Using a heuristic control approach, we evaluate RL's effectiv…

Motion Planning Reinforcement Learning (RL)

Harnessing Structures for Value-Based Planning and Reinforcement Learning

2019-09-26 · ICLR 2020 1 · Yuzhe Yang, Guo Zhang, Zhi Xu, Dina Katabi

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q functi…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1