On the Sample Complexity of Stability Constrained Imitation Learning
We study the following question in the context of imitation learning for continuous control: how are the underlying stability properties of an expert policy reflected in the sample-complexity of an imitation learning task? We provide the first results showing that a surprisingly granular connection can be made between the underlying expert system's incremental gain stability, a novel measure of robust convergence between pairs of system trajectories, and the dependency on the task horizon $T$ of the resulting generalization bounds. In particular, we propose and analyze incremental gain stability constrained versions of behavior cloning and a DAgger-like algorithm, and show that the resulting sample-complexity bounds naturally reflect the underlying stability properties of the expert system. As a special case, we delineate a class of systems for which the number of trajectories needed to achieve $\varepsilon$-suboptimality is sublinear in the task horizon $T$, and do so without requiring (strong) convexity of the loss function in the policy parameters. Finally, we conduct numerical experiments demonstrating the validity of our insights on both a simple nonlinear system for which the underlying stability properties can be easily tuned, and on a high-dimensional quadrupedal robotic simulation.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlGeneralization BoundsImitation LearningSimilar Papers 제목 키워드 기반
KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems
Learning a dynamical system requires stabilizing the unknown dynamics to avoid state blow-ups. However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the …
reinforcement-learningReinforcement Learning (RL)Learning stability guarantees for constrained switching linear systems from noisy observations
We present a data-driven framework based on Lyapunov theory to provide stability guarantees for a family of hybrid systems. In particular, we are interested in the asymptotic stability of switching linear systems whose s…
On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions
When applying imitation learning techniques to fit a policy from expert demonstrations, one can take advantage of prior stability/robustness assumptions on the expert's policy and incorporate such control-theoretic prior…
Imitation LearningBlack-box stability analysis of hybrid systems with sample-based multiple Lyapunov functions
We present a framework based on multiple Lyapunov functions to find probabilistic data-driven guarantees on the stability of unknown constrained switching linear systems (CSLS), which are switching linear systems whose s…
Model Predictive Control via On-Policy Imitation Learning
In this paper, we leverage the rapid advances in imitation learning, a topic of intense recent focus in the Reinforcement Learning (RL) literature, to develop new sample complexity results and performance guarantees for …
Imitation LearningmodelModel Predictive ControlReinforcement Learning (RL)