Skill-based Model-based Reinforcement Learning
Model-based reinforcement learning (RL) is a sample-efficient way of learning complex behaviors by leveraging a learned single-step dynamics model to plan actions in imagination. However, planning every action for long-horizon tasks is not practical, akin to a human planning out every muscle movement. Instead, humans efficiently plan with high-level skills to solve complex tasks. From this intuition, we propose a Skill-based Model-based RL framework (SkiMo) that enables planning in the skill space using a skill dynamics model, which directly predicts the skill outcomes, rather than predicting all small details in the intermediate states, step by step. For accurate and efficient long-term planning, we jointly learn the skill dynamics model and a skill repertoire from prior experience. We then harness the learned skill dynamics model to accurately simulate and plan over long horizons in the skill space, which enables efficient downstream learning of long-horizon, sparse reward tasks. Experimental results in navigation and manipulation domains show that SkiMo extends the temporal horizon of model-based approaches and improves the sample efficiency for both model-based RL and skill-based RL. Code and videos are available at https://clvrai.com/skimo
Code (0)
등록된 구현이 없습니다.
Tasks
modelModel-based Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining
We introduce skill chaining, a skill discovery method for reinforcement learning agents in continuous domains, that builds chains of skills leading to an end-of-task reward. We demonstrate experimentally that it creates …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Skill-Enhanced Reinforcement Learning Acceleration from Demonstrations
Learning from Demonstration (LfD) aims to facilitate rapid Reinforcement Learning (RL) by leveraging expert demonstrations to pre-train the RL agent. However, the limited availability of expert demonstration data often h…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories
We introduce CST, an algorithm for constructing skill trees from demonstration trajectories in continuous reinforcement learning domains. CST uses a changepoint detection method to segment each trajectory into a skill ch…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning
Reinforcement learning provides a general framework for learning robotic skills while minimizing engineering effort. However, most reinforcement learning algorithms assume that a well-designed reward function is provided…
Model Predictive Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
Learning policies for complex tasks that require multiple different skills is a major challenge in reinforcement learning (RL). It is also a requirement for its deployment in real-world scenarios. This paper proposes a n…
Minecraftreinforcement-learningReinforcement LearningReinforcement Learning (RL)