SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks
Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDecision MakingDeep Reinforcement LearningSimilar Papers 제목 키워드 기반
Skill-based Model-based Reinforcement Learning
Model-based reinforcement learning (RL) is a sample-efficient way of learning complex behaviors by leveraging a learned single-step dynamics model to plan actions in imagination. However, planning every action for long-h…
modelModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks. Reinforcement learning (RL) is a promising approach for ac…
Model-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning
Problems which require both long-horizon planning and continuous control capabilities pose significant challenges to existing reinforcement learning agents. In this paper we introduce a novel hierarchical reinforcement l…
continuous-controlContinuous ControlHierarchical Reinforcement Learningreinforcement-learning+2Skill-based Meta-Reinforcement Learning
While deep reinforcement learning methods have shown impressive results in robot learning, their sample inefficiency makes the learning of complex, long-horizon behaviors with real robot systems infeasible. To mitigate t…
continuous-controlContinuous ControlDeep Reinforcement LearningMeta-Learning+4Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and chaining lo…
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1