Robust Model Based Reinforcement Learning Using $\mathcal{L}_1$ Adaptive Control
We introduce $\mathcal{L}_1$-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach generates a series of approximate control-affine models of the learned transition function according to the proposed switching law. Using the approximate model, control input produced by the underlying MBRL is perturbed by the $\mathcal{L}_1$ adaptive control, which is designed to enhance the robustness of the system against uncertainties. Importantly, this approach is agnostic to the choice of MBRL algorithm, enabling the use of the scheme with various MBRL algorithms. MBRL algorithms with $\mathcal{L}_1$ augmentation exhibit enhanced performance and sample efficiency across multiple MuJoCo environments, outperforming the original MBRL algorithms, both with and without system noise.
Code (0)
등록된 구현이 없습니다.
Tasks
Model-based Reinforcement LearningMuJoCoreinforcement-learningSimilar Papers 제목 키워드 기반
Robustifying Reinforcement Learning Policies with $\mathcal{L}_1$ Adaptive Control
A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all env…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)$\mathcal{L}_1$-$\mathcal{GP}$: $\mathcal{L}_1$ Adaptive Control with Bayesian Learning
We present $\mathcal{L}_1$-$\mathcal{GP}$, an architecture based on $\mathcal{L}_1$ adaptive control and Gaussian Process Regression (GPR) for safe simultaneous control and learning. On one hand, the $\mathcal{L}_1$ adap…
GPRregressionData-Driven H-infinity Control with a Real-Time and Efficient Reinforcement Learning Algorithm: An Application to Autonomous Mobility-on-Demand Systems
Reinforcement learning (RL) is a class of artificial intelligence algorithms being used to design adaptive optimal controllers through online learning. This paper presents a model-free, real-time, data-efficient Q-learni…
Q-LearningReinforcement Learning (RL)$\mathcal{L}_1$ Adaptive Augmentation for Geometric Tracking Control of Quadrotors
This paper introduces an $\mathcal{L}_1$ adaptive control augmentation for geometric tracking control of quadrotors. In the proposed design, the $\mathcal{L}_1$ augmentation handles nonlinear (time- and state-dependent) …
On the Optimality, Stability, and Feasibility of Control Barrier Functions: An Adaptive Learning-Based Approach
Safety has been a critical issue for the deployment of learning-based approaches in real-world applications. To address this issue, control barrier function (CBF) and its variants have attracted extensive attention for s…