Stratifying Reinforcement Learning with Signal Temporal Logic

2026-04-06 · Justin Curry, Alberto Speranzon arxiv

In this paper, we develop a stratification-based semantics for Signal Temporal Logic (STL) in which each atomic predicate is interpreted as a membership test in a stratified space. This perspective reveals a novel correspondence principle between stratification theory and STL, showing that most STL formulas can be viewed as inducing a stratification of space-time. The significance of this interpretation is twofold. First, it offers a fresh theoretical framework for analyzing the structure of the embedding space generated by deep reinforcement learning (DRL) and relates it to the geometry of the ambient decision space. Second, it provides a principled framework that both enables the reuse of existing high-dimensional analysis tools and motivates the creation of novel computational techniques. To ground the theory, we (1) illustrate the role of stratification theory in Minigrid games and (2) apply numerical techniques to the latent embeddings of a DRL agent playing such a game where the robustness of STL formulas is used as the reward. In the process, we propose computationally efficient signatures that, based on preliminary evidence, appear promising for uncovering the stratification structure of such embedding spaces.

📄 PDF Abstract BibTeX arXiv:2604.04923

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning

2022-11-30 · Naman Saxena, Gorantla Sandeep, Pushpak Jagtap

Signal Temporal Logic (STL) is a powerful framework for describing the complex temporal and logical behaviour of the dynamical system. Numerous studies have attempted to employ reinforcement learning to learn a controlle…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning from Demonstrations using Signal Temporal Logic

2021-02-15 · Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, Stefanos Nikolaidis

Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is suscepti…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

2022-01-21 · Junya Ikemoto, Toshimitsu Ushio

Deep reinforcement learning (DRL) has attracted much attention as an approach to solve optimal control problems without mathematical models of systems. On the other hand, in general, constraints may be imposed on optimal…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

2024-02-27 · Zijian Guo, Weichao Zhou, Wenchao Li

Offline safe reinforcement learning (RL) aims to train a constraint satisfaction policy from a fixed dataset. Current state-of-the-art approaches are based on supervised learning with a conditioned policy. However, these…

Reinforcement Learning (RL)Safe Reinforcement Learning

Synthesis of Temporally-Robust Policies for Signal Temporal Logic Tasks using Reinforcement Learning

2023-12-10 · Siqi Wang, ShaoYuan Li, Li Yin, Xiang Yin

This paper investigates the problem of designing control policies that satisfy high-level specifications described by signal temporal logic (STL) in unknown, stochastic environments. While many existing works concentrate…

Q-Learning