CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learning

2023-04-07 · Utsav Singh, Vinay P. Namboodiri

Hierarchical reinforcement learning (HRL) is a promising approach that uses temporal abstraction to solve complex long horizon problems. However, simultaneously learning a hierarchy of policies is unstable as it is challenging to train higher-level policy when the lower-level primitive is non-stationary. In this paper, we present CRISP, a novel HRL algorithm that effectively generates a curriculum of achievable subgoals for evolving lower-level primitives using reinforcement learning and imitation learning. CRISP uses the lower level primitive to periodically perform data relabeling on a handful of expert demonstrations, using a novel primitive informed parsing (PIP) approach, thereby mitigating non-stationarity. Since our approach only assumes access to a handful of expert demonstrations, it is suitable for most robotic control tasks. Experimental evaluations on complex robotic maze navigation and robotic manipulation tasks demonstrate that inducing hierarchical curriculum learning significantly improves sample efficiency, and results in efficient goal conditioned policies for solving temporally extended tasks. Additionally, we perform real world robotic experiments on complex manipulation tasks and demonstrate that CRISP demonstrates impressive generalization in real world scenarios.

📄 PDF Abstract BibTeX arXiv:2304.03535

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement LearningImitation Learningreinforcement-learning Reinforcement Learning

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling

2024-04-20 · Utsav Singh, Wesley A. Suttle, Brian M. Sadler, Vinay P. Namboodiri 외

In this work, we introduce PIPER: Primitive-Informed Preference-based Hierarchical reinforcement learning via Hindsight Relabeling, a novel approach that leverages preference-based learning to learn a reward model, and s…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family

2022-10-01 · S Ashwin Hebbar, Viraj Nadkarni, Ashok Vardhan Makkuva, Suma Bhat 외

Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5th generation wireless standards (5G). However, there remains room for the design of polar decoders t…

Decoder

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning

2024-06-16 · Utsav Singh, Souradip Chakraborty, Wesley A. Suttle, Brian M. Sadler 외

Learning control policies to perform complex robotics tasks from human preference data presents significant challenges. On the one hand, the complexity of such tasks typically requires learning policies to perform a vari…

Computational Efficiency Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

2024-06-24 · Vivienne Huiling Wang, Tinghuai Wang, Wenyan Yang, Joni-Kristian Kämäräinen 외

In goal-conditioned hierarchical reinforcement learning (HRL), a high-level policy specifies a subgoal for the low-level policy to reach. Effective HRL hinges on a suitable subgoal represen tation function, abstracting s…

Gaussian Processes Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Subgoal Planning Algorithm for Autonomous Vehicle Guidance

2020-04-27

Trained humans exhibit highly agile spatial skills, enabling them to operate vehicles with complex dynamics in demanding tasks and conditions. Prior work shows that humans achieve this performance by using strategies suc…