Online Algorithms and Policies Using Adaptive and Machine Learning Approaches

2021-05-13 · Anuradha M. Annaswamy, Anubhav Guha, Yingnan Cui, Sunbochen Tang, Peter A. Fisher, Joseph E. Gaudio

This paper considers the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties. We propose a combination of a Reinforcement Learning (RL) based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics, together with Adaptive Control (AC) in the inner loop so that in real-time AC contracts the closed-loop dynamics towards a stable trajectory traced out by RL. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points %with expansion forms around these points and a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform.

📄 PDF Abstract BibTeX arXiv:2105.06577

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Reinforcement Learning (RL)

Online Policies for Real-Time Control Using MRAC-RL

2021-03-30 · Anubhav Guha, Anuradha Annaswamy

In this paper, we propose the Model Reference Adaptive Control & Reinforcement Learning (MRAC-RL) approach to developing online policies for systems in which modeling errors occur in real-time. Although reinforcement lea…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Online Budgeted Learning for Classifier Induction

2019-03-13 · Eran Fainman, Bracha Shapira, Lior Rokach, Yisroel Mirsky

In real-world machine learning applications, there is a cost associated with sampling of different features. Budgeted learning can be used to select which feature-values to acquire from each instance in a dataset, such t…

Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control

2022-06-02 · Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin 외

Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We study…

Adapting multi-armed bandits policies to contextual bandits scenarios

2018-11-11 · David Cortes

This work explores adaptations of successful multi-armed bandits policies to the online contextual bandits scenario with binary rewards using binary classification algorithms such as logistic regression as black-box orac…

Binary Classification Classification General Classification Multi-Armed Bandits+2

MRAC-RL: A Framework for On-Line Policy Adaptation Under Parametric Model Uncertainty

2020-11-20 · Anubhav Guha, Anuradha Annaswamy

Reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems. For many such systems, these policies are trained in a simulated environment. Due to discrepancies bet…