Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specifications

2021-01-18 · Max Cohen, Calin Belta

In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as a sequence of reach-avoid optimal control sub-problems. We show that the resulting hybrid optimal control policy guarantees the satisfaction of a given scLTL formula by constructing a barrier certificate. Since solving each optimal control problem may be computationally intractable, we take a learning-based approach to approximately solve this sequence of optimal control problems online without requiring full knowledge of the system dynamics. Using Lyapunov-based tools, we develop sufficient conditions under which our approximate solution maintains correctness. Finally, we demonstrate the efficacy of the developed method with a numerical example.

📄 PDF Abstract BibTeX arXiv:2101.07156

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Topological Guided Actor-Critic Modular Learning of Continuous Systems with Temporal Objectives

2023-04-20 · Lening Li, Zhentian Qian

This work investigates the formal policy synthesis of continuous-state stochastic dynamic systems given high-level specifications in linear temporal logic. To learn an optimal policy that maximizes the satisfaction proba…

Motion Planning reinforcement-learning Reinforcement Learning

Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specifications

2021-09-27 · Krishna C. Kalagarla, Rahul Jain, Pierluigi Nuzzo

We present a model-free reinforcement learning algorithm to find an optimal policy for a finite-horizon Markov decision process while guaranteeing a desired lower bound on the probability of satisfying a signal temporal …

Motion Planning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

2022-01-21 · Junya Ikemoto, Toshimitsu Ushio

Deep reinforcement learning (DRL) has attracted much attention as an approach to solve optimal control problems without mathematical models of systems. On the other hand, in general, constraints may be imposed on optimal…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learning

2020-03-02 · Abolfazl Lavaei, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi 외

A novel reinforcement learning scheme to synthesize policies for continuous-space Markov decision processes (MDPs) is proposed. This scheme enables one to apply model-free, off-the-shelf reinforcement learning algorithms…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes

2023-05-24 · Krishna C. Kalagarla, Dhruva Kartik, Dongming Shen, Rahul Jain 외

Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is oft…