Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfaction

2020-10-14 · Mingyu Cai, Shaoping Xiao, Baoluo Li, Zhiliang Li, Zhen Kan

This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of linear temporal logic (LTL) specifications. Due to the consideration of environment and motion uncertainties, we model the robot motion as a probabilistic labeled Markov decision process with unknown transition probabilities and unknown probabilistic label functions. The LTL task specification is converted to a limit deterministic generalized B\"uchi automaton (LDGBA) with several accepting sets to maintain dense rewards during learning. The novelty of applying LDGBA is to construct an embedded LDGBA (E-LDGBA) by designing a synchronous tracking-frontier function, which enables the record of non-visited accepting sets without increasing dimensional and computational complexity. With appropriate dependent reward and discount functions, rigorous analysis shows that any method that optimizes the expected discount return of the RL-based approach is guaranteed to find the optimal policy that maximizes the satisfaction probability of the LTL specifications. A model-free RL-based motion planning strategy is developed to generate the optimal policy in this paper. The effectiveness of the RL-based control synthesis is demonstrated via simulation and experimental results.

📄 PDF Abstract BibTeX arXiv:2010.06797

Code (1)

mingyucai/E-LDGBA_RL 공식 구현

Tasks

Motion Planning reinforcement-learning Reinforcement Learning (RL)

Learning from Demonstrations using Signal Temporal Logic

2021-02-15 · Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, Stefanos Nikolaidis

Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is suscepti…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Probabilistic Pontryagin's Maximum Principle for Continuous-Time Model-Based Reinforcement Learning

2025-04-03 · David Leeftink, Çağatay Yıldız, Steffen Ridderbusch, Max Hinne 외

Without exact knowledge of the true system dynamics, optimal control of non-linear continuous-time systems requires careful treatment of epistemic uncertainty. In this work, we propose a probabilistic extension to Pontry…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning

Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees

2019-09-11 · Mohammadhosein Hasanbeig, Yiannis Kantaros, Alessandro Abate, Daniel Kroening 외

Reinforcement Learning (RL) has emerged as an efficient method of choice for solving complex sequential decision making problems in automatic control, computer science, economics, and biology. In this paper we present a …

Decision Making Decision Making Under Uncertainty Hierarchical Reinforcement Learning reinforcement-learning+4

Safeguarding Learning-based Control for Smart Energy Systems with Sampling Specifications

2023-08-11 · Chih-Hong Cheng, Venkatesh Prasad Venkataramanan, Pragya Kirti Gupta, Yun-Fei Hsu 외

We study challenges using reinforcement learning in controlling energy systems, where apart from performance requirements, one has additional safety requirements such as avoiding blackouts. We detail how these safety req…

reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

2018-05-02 · Sergey Levine

The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learnin…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1