ACDER: Augmented Curiosity-Driven Experience Replay

2020-11-16 · Boyao Li, Tao Lu, Jiayi Li, Ning Lu, Yinghao Cai, Shuo Wang

Exploration in environments with sparse feedback remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience Replay (ACDER), which leverages (i) a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully and (ii) the dynamic initial states selection as an automatic exploratory curriculum to further improve the sample-efficiency. Our approach complements Hindsight Experience Replay (HER) by introducing a new way to pursue valuable states. Experiments conducted on four challenging robotic manipulation tasks with binary rewards, including Reach, Push, Pick&Place and Multi-step Push. The empirical results show that our proposed method significantly outperforms existing methods in the first three basic tasks and also achieves satisfactory performance in multi-step robotic task learning.

📄 PDF Abstract BibTeX arXiv:2011.08027

Code (0)

등록된 구현이 없습니다.

Tasks

FetchPush-v1Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Curriculum-guided Hindsight Experience Replay

2019-12-01 · NeurIPS 2019 12 · Meng Fang, Tianyi Zhou, Yali Du, Lei Han 외

In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by t…

Deep Reinforcement Learning Diversity Reinforcement Learning

CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning

2022-05-02 · Chenyu Sun, Hangwei Qian, Chunyan Miao

In reinforcement learning (RL), it is challenging to learn directly from high-dimensional observations, where data augmentation has recently been shown to remedy this via encoding invariances from raw pixels. Nevertheles…

Data Augmentation Q-Learning reinforcement-learning Reinforcement Learning (RL)

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

2019-06-09 · John B. Lanier, Stephen Mcaleer, Pierre Baldi

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as sta…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Curiosity-Driven Experience Prioritization via Density Estimation

2019-02-20 · Rui Zhao, Volker Tresp

In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achi…

Density Estimation OpenAI Gym reinforcement-learning Reinforcement Learning+2

Curious Replay for Model-based Adaptation

2023-06-28 · Isaac Kauvar, Chris Doyle, Linqi Zhou, Nick Haber

Agents must be able to adapt quickly as an environment changes. We find that existing model-based reinforcement learning agents are unable to do this well, in part because of how they use past experiences to train their …

model Model-based Reinforcement Learning