Complex Robotic Manipulation via Graph-Based Hindsight Goal Generation

2020-07-27 · Zhenshan Bing, Matthias Brucker, Fabrice O. Morin, Kai Huang, Alois Knoll

Reinforcement learning algorithms such as hindsight experience replay (HER) and hindsight goal generation (HGG) have been able to solve challenging robotic manipulation tasks in multi-goal settings with sparse rewards. HER achieves its training success through hindsight replays of past experience with heuristic goals, but under-performs in challenging tasks in which goals are difficult to explore. HGG enhances HER by selecting intermediate goals that are easy to achieve in the short term and promising to lead to target goals in the long term. This guided exploration makes HGG applicable to tasks in which target goals are far away from the object's initial position. However, HGG is not applicable to manipulation tasks with obstacles because the euclidean metric used for HGG is not an accurate distance metric in such environments. In this paper, we propose graph-based hindsight goal generation (G-HGG), an extension of HGG selecting hindsight goals based on shortest distances in an obstacle-avoiding graph, which is a discrete representation of the environment. We evaluated G-HGG on four challenging manipulation tasks with obstacles, where significant enhancements in both sample efficiency and overall success rate are shown over HGG and HER. Videos can be viewed at https://sites.google.com/view/demos-g-hgg/.

📄 PDF Abstract BibTeX arXiv:2007.13486

Code (1)

BZSROCKETS/G-HGG tf

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Exploration via Hindsight Goal Generation

2019-06-10 · NeurIPS 2019 12 · Zhizhou Ren, Kefan Dong, Yuan Zhou, Qiang Liu 외

Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sp…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Energy-Based Hindsight Experience Prioritization

2018-10-02 · Rui Zhao, Volker Tresp

In Hindsight Experience Replay (HER), a reinforcement learning agent is trained by treating whatever it has achieved as virtual goals. However, in previous work, the experience was replayed at random, without considering…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

2021-08-17 · Tianhong Dai, Hengyan Liu, Kai Arulkumaran, Guangyu Ren 외

Hindsight experience replay (HER) is a goal relabelling technique typically used with off-policy deep reinforcement learning algorithms to solve goal-oriented tasks; it is well suited to robotic manipulation tasks that d…

Deep Reinforcement Learning Diversity Point Processes

DHER: Hindsight Experience Replay for Dynamic Goals

2019-05-01 · ICLR 2019 5 · Meng Fang, Cheng Zhou, Bei Shi, Boqing Gong 외

Dealing with sparse rewards is one of the most important challenges in reinforcement learning (RL), especially when a goal is dynamic (e.g., to grasp a moving object). Hindsight experience replay (HER) has been shown an …

Object Tracking Reinforcement Learning Reinforcement Learning (RL)

An Open-Source Multi-Goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

2021-05-12 · Xintong Yang, Ze Ji, Jing Wu, Yu-Kun Lai

This work re-implements the OpenAI Gym multi-goal robotic manipulation environment, originally based on the commercial Mujoco engine, onto the open-source Pybullet engine. By comparing the performances of the Hindsight E…

MuJoCo Multi-Goal Reinforcement Learning OpenAI Gym Reinforcement Learning (RL)