Model Primitive Hierarchical Lifelong Reinforcement Learning

2019-03-04 · Bohan Wu, Jayesh K. Gupta, Mykel J. Kochenderfer

Learning interpretable and transferable subpolicies and performing task decomposition from a single, complex task is difficult. Some traditional hierarchical reinforcement learning techniques enforce this decomposition in a top-down manner, while meta-learning techniques require a task distribution at hand to learn such decompositions. This paper presents a framework for using diverse suboptimal world models to decompose complex task solutions into simpler modular subpolicies. This framework performs automatic decomposition of a single source task in a bottom up manner, concurrently learning the required modular subpolicies as well as a controller to coordinate them. We perform a series of experiments on high dimensional continuous action control tasks to demonstrate the effectiveness of this approach at both complex single task learning and lifelong learning. Finally, we perform ablation studies to understand the importance and robustness of different elements in the framework and limitations to this approach.

📄 PDF Abstract BibTeX arXiv:1903.01567

Code (1)

sisl/MPHRL 공식 구현 tf

Tasks

Hierarchical Reinforcement LearningLifelong learningMeta-Learningmodelreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation

2025-04-01 · CVPR 2025 1 · Yuanqi Yao, Siao Liu, Haoming Song, Delin Qu 외

Building a lifelong robot that can effectively leverage prior knowledge for continuous skill acquisition remains significantly challenging. Despite the success of experience replay and parameter-efficient methods in alle…

Prompt Learning Robot Manipulation Transfer Learning

Model-based Lifelong Reinforcement Learning with Bayesian Exploration

2022-10-20 · Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris

We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a samp…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives

2019-06-25 · ICLR 2020 1 · Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng 외

Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior. Often, this is addressed in the context of hierarchical reinforcement learni…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Deep Hierarchical Approach to Lifelong Learning in Minecraft

2016-04-25 · Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J. Mankowitz 외

We propose a lifelong learning system that has the ability to reuse and transfer knowledge from one task to another while efficiently retaining the previously learned knowledge-base. Knowledge is transferred by learning …

Deep Reinforcement Learning Lifelong learning Minecraft Reinforcement Learning

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling

2024-04-20 · Utsav Singh, Wesley A. Suttle, Brian M. Sadler, Vinay P. Namboodiri 외

In this work, we introduce PIPER: Primitive-Informed Preference-based Hierarchical reinforcement learning via Hindsight Relabeling, a novel approach that leverages preference-based learning to learn a reward model, and s…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning