Learning the Linear Quadratic Regulator from Nonlinear Observations
We introduce a new problem setting for continuous control called the LQR with Rich Observations, or RichLQR. In our setting, the environment is summarized by a low-dimensional continuous latent state with linear dynamics and quadratic costs, but the agent operates on high-dimensional, nonlinear observations such as images from a camera. To enable sample-efficient learning, we assume that the learner has access to a class of decoder functions (e.g., neural networks) that is flexible enough to capture the mapping from observations to latent states. We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class. RichID is oracle-efficient and accesses the decoder class only through calls to a least-squares regression oracle. Our results constitute the first provable sample complexity guarantee for continuous control with an unknown nonlinearity in the system model and general function approximation.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDecoderSimilar Papers 제목 키워드 기반
Extracting Latent State Representations with Linear Dynamics from Rich Observations
Recently, many reinforcement learning techniques were shown to have provable guarantees in the simple case of linear dynamics, especially in problems like linear quadratic regulators. However, in practice, many reinforce…
Positionreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Quadratic Control Framework for Dynamic Systems
This article presents a unified approach to quadratic optimal control for both linear and nonlinear discrete-time systems, with a focus on trajectory tracking. The control strategy is based on minimizing a quadratic cost…
Model Predictive ControlPolicy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
Nonlinear control systems with partial information to the decision maker are prevalent in a variety of applications. As a step toward studying such nonlinear systems, this work explores reinforcement learning methods for…
Direct Data-Driven Discrete-time Bilinear Biquadratic Regulator
We present a novel direct data-driven algorithm that learns an optimal control policy for the Bilinear Biquadratic Regulator (BBR) for an unknown bilinear system. The BBR is difficult to solve owing to the presence of th…
Q-LearningQRnet: optimal regulator design with LQR-augmented neural networks
In this paper we propose a new computational method for designing optimal regulators for high-dimensional nonlinear systems. The proposed approach leverages physics-informed machine learning to solve high-dimensional Ham…
BIG-bench Machine LearningPhysics-informed machine learning