A Data-Ensemble-Based Approach for Sample-Efficient LQ Control of Linear Time-Varying Systems
This paper presents a sample-efficient, data-driven control framework for finite-horizon linear quadratic (LQ) control of linear time-varying (LTV) systems. In contrast to the time-invariant case, the time-varying LQ problem involves a differential Riccati equation (DRE) with time-dependent parameters and terminal boundary constraints. We formulate the LQ problem as a nonconvex optimization problem and conduct a rigorous analysis of its dual structure. By exploiting the inherent convexity of the dual problem and analyzing the KKT conditions, we derive an explicit relationship between the optimal dual solution and the parameters of the associated Q-function in time-varying case. This theoretical insight supports the development of a novel, sample-efficient, non-iterative semidefinite programming (SDP) algorithm that directly computes the optimal sequence of feedback gains from an ensemble of input-state data sequences without model identification. The resulting convex, data-dependent framework provides global optimality guarantees for completely unknown LTV systems. As a special case, the method also applies to finite-horizon LQ control of linear time-invariant (LTI) systems. In this setting, a single input-state trajectory suffices to identify the optimal LQ feedback policy, improving significantly over existing Q-learning approaches for finite horizon LTI systems that typically require data from multiple episodes. The approach provides a new optimization-based perspective on Q-learning in time-varying settings and contributes to the broader understanding of data-driven control in non-stationary environments. Simulation results show that, compared to recent methods, the proposed approach achieves superior optimality and sample efficiency on LTV systems, and indicates potential for stabilizing and optimal control of nonlinear systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningSimilar Papers 제목 키워드 기반
ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles
ControlBurn is a Python package to construct feature-sparse tree ensembles that support nonlinear feature selection and interpretable machine learning. The algorithms in this package first build large tree ensembles that…
Additive modelsfeature selectionInterpretable Machine LearningSparse Linear Ensemble Systems and Structural Controllability
The paper introduces and solves a structural controllability problem for continuum ensembles of linear time-invariant systems. All the individual linear systems of an ensemble are sparse, governed by the same sparsity pa…
Controllability Issues of Linear Ensemble Systems over Multi-dimensional Parameterization Spaces
We address an open problem in ensemble control: Whether there exist controllable linear ensemble systems over multi-dimensional parameterization spaces? We provide a negative result: Any real-analytic linear ensemble sys…
Controllability and Tracking of Ensembles: An Optimal Transport Theory Viewpoint
This paper explores the controllability and state tracking of ensembles from the perspective of optimal transport theory. Ensembles, characterized as collections of systems evolving under the same dynamics but with varyi…
Learning-enhanced Nonlinear Model Predictive Control using Knowledge-based Neural Ordinary Differential Equations and Deep Ensembles
Nonlinear model predictive control (MPC) is a flexible and increasingly popular framework used to synthesize feedback control strategies that can satisfy both state and control input constraints. In this framework, an op…
Model Predictive Control