HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learning

2021-08-12 · Willie McClinton, Andrew Levy, George Konidaris

Sparse rewards and long time horizons remain challenging for reinforcement learning algorithms. Exploration bonuses can help in sparse reward settings by encouraging agents to explore the state space, while hierarchical approaches can assist with long-horizon tasks by decomposing lengthy tasks into shorter subtasks. We propose HAC Explore (HACx), a new method that combines these approaches by integrating the exploration bonus method Random Network Distillation (RND) into the hierarchical approach Hierarchical Actor-Critic (HAC). HACx outperforms either component method on its own, as well as an existing approach to combining hierarchy and exploration, in a set of difficult simulated robotics tasks. HACx is the first RL method to solve a sparse reward, continuous-control task that requires over 1,000 actions.

📄 PDF Abstract BibTeX arXiv:2108.05872

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Incremental Hierarchical Reinforcement Learning with Multitask LMDPs

2018-09-27 · Adam C Earle, Andrew M Saxe, Benjamin Rosman

Exploration is a well known challenge in Reinforcement Learning. One principled way of overcoming this challenge is to find a hierarchical abstraction of the base problem and explore at these higher levels, rather than i…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning

2021-10-26 · NeurIPS 2021 12 · Junsu Kim, Younggyo Seo, Jinwoo Shin

Goal-conditioned hierarchical reinforcement learning (HRL) has shown promising results for solving complex and long-horizon RL tasks. However, the action space of high-level policy in the goal-conditioned HRL is often la…

Efficient Exploration Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Learning to Control Visual Abstractions for Structured Exploration in Deep Reinforcement Learning

2019-05-01 · ICLR 2019 5 · catalin ionescu, tejas kulkarni, aaron van de oord, andriy mnih 외

Exploration in environments with sparse rewards is a key challenge for reinforcement learning. How do we design agents with generic inductive biases so that they can explore in a consistent manner instead of just using l…

Atari Games Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning+3

Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning

2025-06-26 · Jaebak Hwang, Sanghyeon Lee, Jeongmo Kim, Seungyul Han

Long-horizon goal-conditioned tasks pose fundamental challenges for reinforcement learning (RL), particularly when goals are distant and rewards are sparse. While hierarchical and graph-based methods offer partial soluti…

Decision Making Hierarchical Reinforcement Learning Reinforcement Learning (RL)

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

2022-10-13 · Shaohui Peng, Xing Hu, Rui Zhang, Ke Tang 외

Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how …

Hierarchical Reinforcement Learning Minecraft reinforcement-learning Reinforcement Learning+1