Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR
We investigate the problem of learning linear quadratic regulators (LQR) in a multi-task, heterogeneous, and model-free setting. We characterize the stability and personalization guarantees of a policy gradient-based (PG) model-agnostic meta-learning (MAML) (Finn et al., 2017) approach for the LQR problem under different task-heterogeneity settings. We show that our MAML-LQR algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias in both model-based and model-free learning scenarios. Moreover, in the model-based setting, we show that such a controller is achieved with a linear convergence rate, which improves upon sub-linear rates from existing work. Our theoretical guarantees demonstrate that the learned controller can efficiently adapt to unseen LQR tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningSimilar Papers 제목 키워드 기반
Model-Agnostic Zeroth-Order Policy Optimization for Meta-Learning of Ergodic Linear Quadratic Regulators
Meta-learning has been proposed as a promising machine learning topic in recent years, with important applications to image classification, robotics, computer games, and control systems. In this paper, we study the probl…
image-classificationImage ClassificationMeta-LearningModel-Agnostic Meta-Policy Optimization via Zeroth-Order Estimation: A Linear Quadratic Regulator Perspective
Meta-learning has been proposed as a promising machine learning topic in recent years, with important applications to image classification, robotics, computer games, and control systems. In this paper, we study the probl…
image-classificationImage ClassificationMeta-LearningOnline Policy Gradient for Model Free Learning of Linear Quadratic Regulators with $\sqrt{T}$ Regret
We consider the task of learning to control a linear dynamical system under fixed quadratic costs, known as the Linear Quadratic Regulator (LQR) problem. While model-free approaches are often favorable in practice, thus …
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
Nonlinear control systems with partial information to the decision maker are prevalent in a variety of applications. As a step toward studying such nonlinear systems, this work explores reinforcement learning methods for…
On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator
We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)