AUTOMATA GUIDED HIERARCHICAL REINFORCEMENT LEARNING FOR ZERO-SHOT SKILL COMPOSITION

2018-01-01 · ICLR 2018 1 · Xiao Li, Yao Ma, Calin Belta

An obstacle that prevents the wide adoption of (deep) reinforcement learning (RL) in control systems is its need for a large number of interactions with the environment in order to master a skill. The learned skill usually generalizes poorly across domains and re-training is often necessary when presented with a new task. We present a framework that combines techniques in \textit{formal methods} with \textit{hierarchical reinforcement learning} (HRL). The set of techniques we provide allows for the convenient specification of tasks with logical expressions, learns hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards using any RL methods and is able to construct new skills from existing ones without additional learning. We evaluate the proposed methods in a simple grid world simulation as well as simulation on a Baxter robot.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningHierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learning

2024-10-31 · Beyazit Yalcinkaya, Niklas Lauffer, Marcell Vazquez-Chanlatte, Sanjit A. Seshia

Goal-conditioned reinforcement learning is a powerful way to control an AI agent's behavior at runtime. That said, popular goal representations, e.g., target states or natural language, are either limited to Markovian ta…

Graph Neural Network reinforcement-learning Reinforcement Learning Zero-shot Generalization

Automata-Guided Hierarchical Reinforcement Learning for Skill Composition

2017-10-31 · Xiao Li, Yao Ma, Calin Belta

Skills learned through (deep) reinforcement learning often generalizes poorly across domains and re-training is necessary when presented with a new task. We present a framework that combines techniques in \textit{formal …

Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Automata Guided Reinforcement Learning With Demonstrations

2018-09-17 · Xiao Li, Yao Ma, Calin Belta

Tasks with complex temporal structures and long horizons pose a challenge for reinforcement learning agents due to the difficulty in specifying the tasks in terms of reward functions as well as large variances in the lea…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning

2025-08-03 · Zijian Guo, İlker Işık, H. M. Sabbir Ahmad, Wenchao Li arxiv

Generalizing to complex and temporally extended task objectives and safety constraints remains a critical challenge in reinforcement learning (RL). Linear temporal logic (LTL) offers a unified formalism to specify such r…

Zero-shot Generalization Reinforcement Learning

Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learning

2026-05-13 · Stefan Stojanovic, Alexandre Proutiere arxiv

Hierarchical reinforcement learning can improve generalization by decomposing long-horizon decision-making into simpler subproblems. However, existing approaches often rely on restrictive design choices, such as fixed te…

Hierarchical Reinforcement Learning