Curiosity-Driven Experience Prioritization via Density Estimation

2019-02-20 · Rui Zhao, Volker Tresp

In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised learning, but has not yet been thoroughly researched in RL. To address this problem, we propose a novel Curiosity-Driven Prioritization (CDP) framework to encourage the agent to over-sample those trajectories that have rare achieved goal states. The CDP framework mimics the human learning process and focuses more on relatively uncommon events. We evaluate our methods using the robotic environment provided by OpenAI Gym. The environment contains six robot manipulation tasks. In our experiments, we combined CDP with Deep Deterministic Policy Gradient (DDPG) with or without Hindsight Experience Replay (HER). The experimental results show that CDP improves both performance and sample-efficiency of reinforcement learning agents, compared to state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:1902.08039

Code (0)

등록된 구현이 없습니다.

Tasks

Density Estimation OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Robot Manipulation

ARTPS: Depth-Enhanced Hybrid Anomaly Detection and Learnable Curiosity Score for Autonomous Rover Target Prioritization

2025-08-23 · Poyraz Baydemir arxiv

We present ARTPS (Autonomous Rover Target Prioritization System), a novel hybrid AI system that combines depth estimation, anomaly detection, and learnable curiosity scoring for autonomous exploration of planetary surfac…

Monocular Depth Estimation Anomaly Detection

Curriculum-guided Hindsight Experience Replay

2019-12-01 · NeurIPS 2019 12 · Meng Fang, Tianyi Zhou, Yali Du, Lei Han 외

In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by t…

Deep Reinforcement Learning Diversity Reinforcement Learning

ACDER: Augmented Curiosity-Driven Experience Replay

2020-11-16 · Boyao Li, Tao Lu, Jiayi Li, Ning Lu 외

Exploration in environments with sparse feedback remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, espec…

FetchPush-v1 Reinforcement Learning (RL)

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

2019-06-09 · John B. Lanier, Stephen Mcaleer, Pierre Baldi

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as sta…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Experience Replay with Likelihood-free Importance Weights

2020-06-23 · Samarth Sinha, Jiaming Song, Animesh Garg, Stefano Ermon

The use of past experiences to accelerate temporal difference (TD) learning of value functions, or experience replay, is a key component in deep reinforcement learning. Prioritization or reweighting of important experien…

Deep Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning (RL)