Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics
Automated synthesis of provably correct controllers for cyber-physical systems is crucial for deployment in safety-critical scenarios. However, hybrid features and stochastic or unknown behaviours make this problem challenging. We propose a method for synthesising controllers for Markov jump linear systems (MJLSs), a class of discrete-time models for cyber-physical systems, so that they certifiably satisfy probabilistic computation tree logic (PCTL) formulae. An MJLS consists of a finite set of stochastic linear dynamics and discrete jumps between these dynamics that are governed by a Markov decision process (MDP). We consider the cases where the transition probabilities of this MDP are either known up to an interval or completely unknown. Our approach is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We formalise this abstraction as an interval MDP (iMDP) for which we compute intervals of transition probabilities using sampling techniques from the so-called 'scenario approach', resulting in a probabilistically sound approximation. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Sampled-Data Controller Synthesis using Dissipative Linear Periodic Jump-Flow Systems with Design Applications
In this paper, we will propose linear-matrix-inequality-based techniques for the design of sampled-data controllers that render the closed-loop system dissipative with respect to \textcolor{blue}{quadratic supply functio…
Policy Optimization for Markovian Jump Linear Quadratic Control: Gradient-Based Methods and Global Convergence
Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. In this paper, we investigate the global convergence of gradient-based policy op…
Policy Gradient MethodsModel-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach
This research paper introduces a model-free optimal controller for discrete-time Markovian jump linear systems (MJLSs), employing principles from the methodology of reinforcement learning (RL). While Q-learning methods h…
Q-LearningReinforcement Learning (RL)Predictive Control of Linear Discrete-Time Markovian Jump Systems by Learning Recurrent Patterns
Incorporating pattern-learning for prediction (PLP) in many discrete-time or discrete-event systems allows for computation-efficient controller design by memorizing patterns to schedule control policies based on their fu…
Convergence Guarantees of Policy Optimization Methods for Markovian Jump Linear Systems
Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. In this paper, we investigate the convergence of policy optimization for quadrat…
Reinforcement Learning