Modular Deep Reinforcement Learning with Temporal Logic Specifications

2019-09-23 · Lim Zun Yuan, Mohammadhosein Hasanbeig, Alessandro Abate, Daniel Kroening

We propose an actor-critic, model-free, and online Reinforcement Learning (RL) framework for continuous-state continuous-action Markov Decision Processes (MDPs) when the reward is highly sparse but encompasses a high-level temporal structure. We represent this temporal structure by a finite-state machine and construct an on-the-fly synchronised product with the MDP and the finite machine. The temporal structure acts as a guide for the RL agent within the product, where a modular Deep Deterministic Policy Gradient (DDPG) architecture is proposed to generate a low-level control policy. We evaluate our framework in a Mars rover experiment and we present the success rate of the synthesised policy.

📄 PDF Abstract BibTeX arXiv:1909.11591

Code (2)

RickyMexx/DeepRL-LTL pytorch

RickyMexx/DeepRL-LTLf pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning with Temporal Logic Specifications

2021-02-01 · Lewis Hammond, Alessandro Abate, Julian Gutierrez, Michael Wooldridge

In this paper, we study the problem of learning to satisfy temporal logic specifications with a group of agents in an unknown environment, which may exhibit probabilistic behaviour. From a learning perspective these spec…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Modularized Control Synthesis for Complex Signal Temporal Logic Specifications

2023-03-30 · Zengjie Zhang, Sofie Haesaert

The control synthesis of a dynamic system subject to a signal temporal logic (STL) specification is commonly formulated as a mixed-integer linear/convex programming (MILP/MICP) problem. Solving such a problem is computat…

LTLGuard: Formalizing LTL Specifications with Compact Language Models and Lightweight Symbolic Reasoning

2026-03-05 · Medina Andresel, Cristinel Mateis, Dejan Nickovic, Spyridon Kounoupidis 외 arxiv

Translating informal requirements into formal specifications is challenging due to the ambiguity and variability of natural language (NL). This challenge is particularly pronounced when relying on compact (small and medi…

Topological Guided Actor-Critic Modular Learning of Continuous Systems with Temporal Objectives

2023-04-20 · Lening Li, Zhentian Qian

This work investigates the formal policy synthesis of continuous-state stochastic dynamic systems given high-level specifications in linear temporal logic. To learn an optimal policy that maximizes the satisfaction proba…

Motion Planning reinforcement-learning Reinforcement Learning

Learning from Demonstrations using Signal Temporal Logic

2021-02-15 · Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, Stefanos Nikolaidis

Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is suscepti…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)