Curious Hierarchical Actor-Critic Reinforcement Learning
Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-critic approach with a forward model to develop a hierarchical notion of curiosity. We demonstrate in several continuous-space environments that curiosity can more than double the learning performance and success rates for most of the investigated benchmarking problems. We also provide our source code and a supplementary video.
Code (1)
Tasks
BenchmarkingHierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments
Reinforcement Learning (RL) is known to be often unsuccessful in environments with sparse extrinsic rewards. A possible countermeasure is to endow RL agents with an intrinsic reward function, or 'intrinsic motivation', w…
Reinforcement Learning (RL)Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning
Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first …
Hierarchical Reinforcement LearningContinuous ControlQuantum Hierarchical Reinforcement Learning via Variational Quantum Circuits
Reinforcement learning is one of the most challenging learning paradigms where efficacy and efficiency gains are extremely valuable. Hierarchical reinforcement learning is a variant that leverages temporal abstraction to…
Hierarchical Reinforcement LearningHybrid Actor-Critic Reinforcement Learning in Parameterized Action Space
In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured actio…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Curious iLQR: Resolving Uncertainty in Model-based RL
Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a…
modelModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1