Safe Pontryagin Differentiable Programming
We propose a Safe Pontryagin Differentiable Programming (Safe PDP) methodology, which establishes a theoretical and algorithmic framework to solve a broad class of safety-critical learning and control tasks -- problems that require the guarantee of safety constraint satisfaction at any stage of the learning and control progress. In the spirit of interior-point methods, Safe PDP handles different types of system constraints on states and inputs by incorporating them into the cost or loss through barrier functions. We prove three fundamentals of the proposed Safe PDP: first, both the solution and its gradient in the backward pass can be approximated by solving their more efficient unconstrained counterparts; second, the approximation for both the solution and its gradient can be controlled for arbitrary accuracy by a barrier parameter; and third, importantly, all intermediate results throughout the approximation and optimization strictly respect the constraints, thus guaranteeing safety throughout the entire learning and control process. We demonstrate the capabilities of Safe PDP in solving various safety-critical tasks, including safe policy optimization, safe motion planning, and learning MPCs from demonstrations, on different challenging systems such as 6-DoF maneuvering quadrotor and 6-DoF rocket powered landing.
Code (1)
Tasks
Motion PlanningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
This paper develops a Pontryagin Differentiable Programming (PDP) methodology, which establishes a unified framework to solve a broad class of learning and control tasks. The PDP distinguishes from existing methods by tw…
Pontryagin Neural Operator for Solving Parametric General-Sum Differential Games
The values of two-player general-sum differential games are viscosity solutions to Hamilton-Jacobi-Isaacs (HJI) equations. Value and policy approximations for such games suffer from the curse of dimensionality (CoD). All…
Pontryagin-Guided Deep Learning for Large-Scale Constrained Dynamic Portfolio Choice
We present a Pontryagin-Guided Direct Policy Optimization (PG-DPO) method for constrained dynamic portfolio choice - incorporating consumption and multi-asset investment - that scales to thousands of risky assets. By com…
Deep LearningPortfolio OptimizationBeyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting
Most value-based and actor--critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We s…
Reinforcement LearningExtended Kalman Filtering for Recursive Online Discrete-Time Inverse Optimal Control
We formulate the discrete-time inverse optimal control problem of inferring unknown parameters in the objective function of an optimal control problem from measurements of optimal states and controls as a nonlinear filte…