Creating Multi-Level Skill Hierarchies in Reinforcement Learning

2023-06-16 · NeurIPS 2023 11 · Joshua B. Evans, Özgür Şimşek

What is a useful skill hierarchy for an autonomous agent? We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction. The result is a collection of skills that operate at varying time scales, organised into a hierarchy, where skills that operate over longer time scales are composed of skills that operate over shorter time scales. The entire skill hierarchy is generated automatically, with no human intervention, including the skills themselves (their behaviour, when they can be called, and when they terminate) as well as the hierarchical dependency structure between them. In a wide range of environments, this approach generates skill hierarchies that are intuitively appealing and that considerably improve the learning performance of the agent.

📄 PDF Abstract BibTeX arXiv:2306.09980

Code (1)

bath-reinforcement-learning-lab/louvain-skills-neurips-2023 공식 구현

Tasks

graph partitioning reinforcement-learning Reinforcement Learning

Accelerating Task Generalisation with Multi-Level Skill Hierarchies

2024-11-05 · Thomas P Cannon, Özgür Simsek

Creating reinforcement learning agents that generalise effectively to new tasks is a key challenge in AI research. This paper introduces Fracture Cluster Options (FraCOs), a multi-level hierarchical reinforcement learnin…

Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Unsupervised Hierarchical Skill Discovery

2026-01-30 · Damion Harvey, Geraud Nangue Tasse, Benjamin Rosman, Branden Ingram 외 arxiv

We consider the problem of unsupervised skill segmentation and hierarchical structure discovery in reinforcement learning. While recent approaches have sought to segment trajectories into reusable skills or options, most…

Reinforcement Learning

Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning

2021-11-04 · ICLR 2022 4 · Dhruv Shah, Peng Xu, Yao Lu, Ted Xiao 외

Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and chaining lo…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery

2019-12-07 · Jiachen Yang, Igor Borovikov, Hongyuan Zha

Human players in professional team sports achieve high level coordination by dynamically choosing complementary skills and executing primitive actions to perform these skills. As a step toward creating intelligent agents…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Modulated Policy Hierarchies

2018-11-30 · Alexander Pashevich, Danijar Hafner, James Davidson, Rahul Sukthankar 외

Solving tasks with sparse rewards is a main challenge in reinforcement learning. While hierarchical controllers are an intuitive approach to this problem, current methods often require manual reward shaping, alternating …

Reinforcement Learning Reinforcement Learning (RL)