On the Computational Consequences of Cost Function Design in Nonlinear Optimal Control
Optimal control is an essential tool for stabilizing complex nonlinear systems. However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error. In this paper we seek to gain insights into how the choice of cost function interacts with the underlying structure of the control system and impacts the amount of computation required to obtain a stabilizing controller. We treat the cost design problem as a two-step process where the designer specifies outputs for the system that are to be penalized and then modulates the relative weighting of the inputs and the outputs in the cost. To characterize the computational burden associated to obtaining a stabilizing controller with a particular cost, we bound the prediction horizon required by receding horizon methods and the number of iterations required by dynamic programming methods to meet this requirement. Our theoretical results highlight a qualitative separation between what is possible, from a design perspective, when the chosen outputs induce either minimum-phase or non-minimum-phase behavior. Simulation studies indicate that this separation also holds for modern reinforcement learning methods.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Basis functions nonlinear data-enabled predictive control: Consistent and computationally efficient formulations
This paper considers the extension of data-enabled predictive control (DeePC) to nonlinear systems via general basis functions. Firstly, we formulate a basis functions DeePC behavioral predictor and we identify necessary…
Computational EfficiencyForce-Limited Control of Wave Energy Converters using a Describing Function Linearization
Actuator saturation is a common nonlinearity. In wave energy conversion, force saturation conveniently limits drivetrain size and cost with minimal impact on energy generation. However, such nonlinear dynamics typically …
Robust Quadratic Gaussian Control of Continuous-time Nonlinear Systems
In this paper, we propose a new Robust Nonlinear Quadratic Gaussian (RNQG) controller based on State-Dependent Riccati Equation (SDRE) scheme for continuous-time nonlinear systems. Existing controllers do not account for…
Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions
Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains difficult for nonlinear and uncertain models. We develop a model-based learning approach to sy…
Enforcing contraction via data
We present data-based conditions for enforcing contractivity via feedback control and obtain desired asymptotic properties of the closed-loop system. We focus on unknown nonlinear control systems whose vector fields are …