Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control

2021-12-11 · F. De Lellis, M. Coraggio, G. Russo, M. Musolesi, M. di Bernardo

We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem.

📄 PDF Abstract BibTeX arXiv:2112.06018

Code (0)

등록된 구현이 없습니다.

Tasks

OpenAI GymQ-LearningReinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Control-Tutored Reinforcement Learning: an application to the Herding Problem

2019-11-26 · Francesco De Lellis, Fabrizia Auletta, Giovanni Russo, Mario di Bernardo

In this extended abstract we introduce a novel control-tutored Q-learning approach (CTQL) as part of the ongoing effort in developing model-based and safe RL for continuous state spaces. We validate our approach by apply…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Control-Tutored Reinforcement Learning

2019-12-12 · Francesco De Lellis, Fabrizia Auletta, Giovanni Russo, Piero De Lellis 외

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms so as to improve the exploration of the state-space, and substantially reduce learning times by l…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

CT-DQN: Control-Tutored Deep Reinforcement Learning

2022-12-02 · Francesco De Lellis, Marco Coraggio, Giovanni Russo, Mirco Musolesi 외

One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algo…

Car Racing Deep Reinforcement Learning OpenAI Gym reinforcement-learning+2

Tutoring Reinforcement Learning via Feedback Control

2020-12-12 · Francesco De Lellis, Giovanni Russo, Mario di Bernardo

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms by means of a control strategy with limited knowledge of the system model. By tutoring the learni…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A modular framework for stabilizing deep reinforcement learning control

2023-04-07 · Nathan P. Lawrence, Philip D. Loewen, Shuyuan Wang, Michael G. Forbes 외

We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kucer…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning