Accelerating Task Generalisation with Multi-Level Skill Hierarchies
Creating reinforcement learning agents that generalise effectively to new tasks is a key challenge in AI research. This paper introduces Fracture Cluster Options (FraCOs), a multi-level hierarchical reinforcement learning method that achieves state-of-the-art performance on difficult generalisation tasks. FraCOs identifies patterns in agent behaviour and forms options based on the expected future usefulness of those patterns, enabling rapid adaptation to new tasks. In tabular settings, FraCOs demonstrates effective transfer and improves performance as it grows in hierarchical depth. We evaluate FraCOs against state-of-the-art deep reinforcement learning algorithms in several complex procedurally generated environments. Our results show that FraCOs achieves higher in-distribution and out-of-distribution performance than competitors.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningHierarchical Reinforcement Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics
Skill-based reinforcement learning (RL) has emerged as a promising strategy to leverage prior knowledge for accelerated robot learning. Skills are typically extracted from expert demonstrations and are embedded into a la…
Reinforcement Learning (RL)Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learning
Meta-Reinforcement Learning (Meta-RL) agents can struggle to operate across tasks with varying environmental features that require different optimal skills (i.e., different modes of behaviour). Using context encoders bas…
Contrastive LearningMeta Reinforcement LearningMuJoCoreinforcement-learning+1Combining Modular Skills in Multitask Learning
A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume that each task is associated with a subset of l…
Instruction Followingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Skill-Enhanced Reinforcement Learning Acceleration from Demonstrations
Learning from Demonstration (LfD) aims to facilitate rapid Reinforcement Learning (RL) by leveraging expert demonstrations to pre-train the RL agent. However, the limited availability of expert demonstration data often h…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Successes and Limitations of Object-centric Models at Compositional Generalisation
In recent years, it has been shown empirically that standard disentangled latent variable models do not support robust compositional learning in the visual domain. Indeed, in spite of being designed with the goal of fact…