Safe Q-learning for continuous-time linear systems
Q-learning is a promising method for solving optimal control problems for uncertain systems without the explicit need for system identification. However, approaches for continuous-time Q-learning have limited provable safety guarantees, which restrict their applicability to real-time safety-critical systems. This paper proposes a safe Q-learning algorithm for partially unknown linear time-invariant systems to solve the linear quadratic regulator problem with user-defined state constraints. We frame the safe Q-learning problem as a constrained optimal control problem using reciprocal control barrier functions and show that such an extension provides a safety-assured control policy. To the best of our knowledge, Q-learning for continuous-time systems with state constraints has not yet been reported in the literature.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Safe reinforcement learning control for continuous-time nonlinear systems without a backup controller
This paper proposes an on-policy reinforcement learning (RL) control algorithm that solves the optimal regulation problem for a class of uncertain continuous-time nonlinear systems under user-defined state constraints. W…
reinforcement-learningReinforcement Learning (RL)Safe Reinforcement LearningPRoTECT: Parallelized Construction of Safety Barrier Certificates for Nonlinear Polynomial Systems
We develop an open-source software tool, called PRoTECT, for the parallelized construction of safety barrier certificates (BCs) for nonlinear polynomial systems. This tool employs sum-of-squares (SOS) optimization progra…
Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models
Control Barrier Functions (CBFs) have been demonstrated to be a powerful tool for safety-critical controller design for nonlinear systems. Existing design paradigms do not address the gap between theory (controller desig…
TRUST: Stability and Safety Controller Synthesis for Unknown Dynamical Models Using a Single Trajectory
TRUST is an open-source software tool developed for data-driven controller synthesis of dynamical systems with unknown mathematical models, ensuring either stability or safety properties. By collecting only a single inpu…
LEMMASafety Verification of Decision-Tree Policies in Continuous Time
Decision trees have gained popularity as interpretable surrogate models for learning-based control policies. However, providing safety guarantees for systems controlled by decision trees is an open challenge. We show tha…