Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks

2024-10-03 · Zeyu Feng, Hao Luan, Kevin Yuchen Ma, Harold Soh

Safe and successful deployment of robots requires not only the ability to generate complex plans but also the capacity to frequently replan and correct execution errors. This paper addresses the challenge of long-horizon trajectory planning under temporally extended objectives in a receding horizon manner. To this end, we propose DOPPLER, a data-driven hierarchical framework that generates and updates plans based on instruction specified by linear temporal logic (LTL). Our method decomposes temporal tasks into chain of options with hierarchical reinforcement learning from offline non-expert datasets. It leverages diffusion models to generate options with low-level actions. We devise a determinantal-guided posterior sampling technique during batch generation, which improves the speed and diversity of diffusion generated options, leading to more efficient querying. Experiments on robot navigation and manipulation tasks demonstrate that DOPPLER can generate sequences of trajectories that progressively satisfy the specified formulae for obstacle avoidance and sequential visitation. Demonstration videos are available online at: https://philiptheother.github.io/doppler/.

📄 PDF Abstract BibTeX arXiv:2410.02389

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityHierarchical Reinforcement Learning Robot Navigation Trajectory Planning

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Joint Learning of Hierarchical Neural Options and Abstract World Model

2026-02-02 · Wasu Top Piriyakulkij, Wolfgang Lehrach, Kevin Ellis, Kevin Murphy arxiv

Building agents that can perform new skills by composing existing skills is a long-standing goal of AI agent research. Towards this end, we investigate how to efficiently acquire a sequence of skills, formalized as hiera…

Atari Games

Adversarial Option-Aware Hierarchical Imitation Learning

2021-06-10 · Mingxuan Jing, Wenbing Huang, Fuchun Sun, Xiaojian Ma 외

It has been a challenge to learning skills for an agent from long-horizon unannotated demonstrations. Existing approaches like Hierarchical Imitation Learning(HIL) are prone to compounding errors or suboptimal solutions.…

Imitation Learning

The Option Keyboard: Combining Skills in Reinforcement Learning

2021-06-24 · NeurIPS 2019 12 · André Barreto, Diana Borsa, Shaobo Hou, Gheorghe Comanici 외

The ability to combine known skills to create new ones may be crucial in the solution of complex reinforcement learning problems that unfold over extended periods. We argue that a robust way of combining skills is to def…

Management reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Successor Options : An Option Discovery Algorithm for Reinforcement Learning

2018-09-27 · Manan Tomar*, Rahul Ramesh*, Balaraman Ravindran

Hierarchical Reinforcement Learning is a popular method to exploit temporal abstractions in order to tackle the curse of dimensionality. The options framework is one such hierarchical framework that models the notion of …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning

2019-09-09 · Arjun Manoharan, Rahul Ramesh, Balaraman Ravindran

Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent. However, such techniques often yield a large number of options or skills, whic…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)