Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized Büchi Automata

2020-01-14 · Ryohei Oura, Ami Sakakibara, Toshimitsu Ushio

This letter proposes a novel reinforcement learning method for the synthesis of a control policy satisfying a control specification described by a linear temporal logic formula. We assume that the controlled system is modeled by a Markov decision process (MDP). We convert the specification to a limit-deterministic generalized B\"uchi automaton (LDGBA) with several accepting sets that accepts all infinite sequences satisfying the formula. The LDGBA is augmented so that it explicitly records the previous visits to accepting sets. We take a product of the augmented LDGBA and the MDP, based on which we define a reward function. The agent gets rewards whenever state transitions are in an accepting set that has not been visited for a certain number of steps. Consequently, sparsity of rewards is relaxed and optimal circulations among the accepting sets are learned. We show that the proposed method can learn an optimal policy when the discount factor is sufficiently close to one.

📄 PDF Abstract BibTeX arXiv:2001.04669

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning

2019-09-16 · Alper Kamil Bozkurt, Yu Wang, Michael M. Zavlanos, Miroslav Pajic

We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Proce…

Motion Planning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Accelerated Reinforcement Learning for Temporal Logic Control Objectives

2022-05-09 · Yiannis Kantaros

This paper addresses the problem of learning control policies for mobile robots, modeled as unknown Markov Decision Processes (MDPs), that are tasked with temporal logic missions, such as sequencing, coverage, or surveil…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specifications

2021-01-18 · Max Cohen, Calin Belta

In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as …

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Signal Temporal Logic Neural Predictive Control

2023-09-10 · Yue Meng, Chuchu Fan

Ensuring safety and meeting temporal specifications are critical challenges for long-term robotic tasks. Signal temporal logic (STL) has been widely used to systematically and rigorously specify these requirements. Howev…

Model Predictive Control Reinforcement Learning (RL)

Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfaction

2020-10-14 · Mingyu Cai, Shaoping Xiao, Baoluo Li, Zhiliang Li 외

This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of linear temporal logic (LTL) specifications. Due to the consideration of…

Motion Planning reinforcement-learning Reinforcement Learning (RL)