Instructing Goal-Conditioned Reinforcement Learning Agents with Temporal Logic Objectives

2023-09-21 · NeurIPS 2023 11

Goal-conditioned reinforcement learning (RL) is a powerful approach for learning general-purpose skills by reaching diverse goals. However, it has limitations when it comes to task-conditioned policies, where goals are specified by temporally extended instructions written in the Linear Temporal Logic (LTL) formal language. Existing approaches for finding LTL-satisfying policies rely on sampling a large set of LTL instructions during training to adapt to unseen tasks at inference time. However, these approaches do not guarantee generalization to out-of-distribution LTL objectives, which may have increased complexity. In this paper, we propose a novel approach to address this challenge. We show that simple goal-conditioned RL agents can be instructed to follow arbitrary LTL specifications without additional training over the LTL task space. Unlike existing approaches that focus on LTL specifications expressible as regular expressions, our technique is unrestricted and generalizes to $\omega$-regular expressions. Experiment results demonstrate the effectiveness of our approach in adapting goal-conditioned RL agents to satisfy complex temporal logic task specifications zero-shot.Submission Number: 14481

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Focus 설명 없음

TEACH: Temporal Variance-Driven Curriculum for Reinforcement Learning

2025-12-28 · Gaurav Chaudhary, Laxmidhar Behera arxiv

Reinforcement Learning (RL) has achieved significant success in solving single-goal tasks. However, uniform goal selection often results in sample inefficiency in multi-goal settings where agents must learn a universal g…

Reinforcement Learning

Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

2023-07-20 · Zhifeng Qian, Mingyu You, Hongjun Zhou, Xuanhui Xu 외

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally e…

Disentanglement reinforcement-learning Reinforcement Learning

Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learning

2025-02-11 · Qingyuan Wu, Jianheng Liu, Jianye Hao, Jun Wang 외

State-of-the-art (SOTA) reinforcement learning (RL) methods have enabled vision-language model (VLM) agents to learn from interaction with online environments without human supervision. However, these methods often strug…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learning

2024-10-31 · Beyazit Yalcinkaya, Niklas Lauffer, Marcell Vazquez-Chanlatte, Sanjit A. Seshia

Goal-conditioned reinforcement learning is a powerful way to control an AI agent's behavior at runtime. That said, popular goal representations, e.g., target states or natural language, are either limited to Markovian ta…

Graph Neural Network reinforcement-learning Reinforcement Learning Zero-shot Generalization

Goal-Conditioned Reinforcement Learning in the Presence of an Adversary

2022-11-13 · Carlos Purves, Pietro Liò, Cătălina Cangea

Reinforcement learning has seen increasing applications in real-world contexts over the past few years. However, physical environments are often imperfect and policies that perform well in simulation might not achieve th…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)