Cluster-based Sampling in Hindsight Experience Replay for Robotic Tasks (Student Abstract)

2022-08-31 · TaeYoung Kim, Dongsoo Har

In multi-goal reinforcement learning with a sparse binary reward, training agents is particularly challenging, due to a lack of successful experiences. To solve this problem, hindsight experience replay (HER) generates successful experiences even from unsuccessful ones. However, generating successful experiences from uniformly sampled ones is not an efficient process. In this paper, the impact of exploiting the property of achieved goals in generating successful experiences is investigated and a novel cluster-based sampling strategy is proposed. The proposed sampling strategy groups episodes with different achieved goals by using a cluster model and samples experiences in the manner of HER to create the training batch. The proposed method is validated by experiments with three robotic control tasks of the OpenAI Gym. The results of experiments demonstrate that the proposed method is substantially sample efficient and achieves better performance than baseline approaches.

📄 PDF Abstract BibTeX arXiv:2208.14741

Code (0)

등록된 구현이 없습니다.

Tasks

ClusteringMulti-Goal Reinforcement Learning OpenAI Gym

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

k-Means Clustering k-Means Clustering is a clustering algorithm that divides a training set into $k$ different clusters of examples that are near each other. It works by initializing $k$…

Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environment

2021-10-28 · Tung M. Luu, Chang D. Yoo

This paper proposes a method for prioritizing the replay experience referred to as Hindsight Goal Ranking (HGR) in overcoming the limitation of Hindsight Experience Replay (HER) that generates hindsight goals based on un…

ARCHER: Aggressive Rewards to Counter bias in Hindsight Experience Replay

2018-09-06 · Sameera Lanka, Tianfu Wu

Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful…

continuous-control Continuous Control Deep Reinforcement Learning Reinforcement Learning+1

USHER: Unbiased Sampling for Hindsight Experience Replay

2022-07-03 · Liam Schramm, Yunfu Deng, Edgar Granados, Abdeslam Boularias

Dealing with sparse rewards is a long-standing challenge in reinforcement learning (RL). Hindsight Experience Replay (HER) addresses this problem by reusing failed trajectories for one goal as successful trajectories for…

reinforcement-learning Reinforcement Learning (RL)

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

2021-08-17 · Tianhong Dai, Hengyan Liu, Kai Arulkumaran, Guangyu Ren 외

Hindsight experience replay (HER) is a goal relabelling technique typically used with off-policy deep reinforcement learning algorithms to solve goal-oriented tasks; it is well suited to robotic manipulation tasks that d…

Deep Reinforcement Learning Diversity Point Processes

Energy-Based Hindsight Experience Prioritization

2018-10-02 · Rui Zhao, Volker Tresp

In Hindsight Experience Replay (HER), a reinforcement learning agent is trained by treating whatever it has achieved as virtual goals. However, in previous work, the experience was replayed at random, without considering…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)