Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

2023-09-18 · Mohammad S. Ramadan, Mahmoud A. Hayajnh, Michael T. Tolley, Kyriakos G. Vamvoudakis

In this paper we propose a framework towards achieving two intertwined objectives: (i) equipping reinforcement learning with active exploration and deliberate information gathering, such that it regulates state and parameter uncertainties resulting from modeling mismatches and noisy sensory; and (ii) overcoming the computational intractability of stochastic optimal control. We approach both objectives by using reinforcement learning to compute the stochastic optimal control law. On one hand, we avoid the curse of dimensionality prohibiting the direct solution of the stochastic dynamic programming equation. On the other hand, the resulting stochastic optimal control reinforcement learning agent admits caution and probing, that is, optimal online exploration and exploitation. Unlike fixed exploration and exploitation balance, caution and probing are employed automatically by the controller in real-time, even after the learning process is terminated. We conclude the paper with a numerical simulation, illustrating how a Linear Quadratic Regulator with the certainty equivalence assumption may lead to poor performance and filter divergence, while our proposed approach is stabilizing, of an acceptable performance, and computationally convenient.

📄 PDF Abstract BibTeX arXiv:2309.10831

Code (1)

msramada/active-learning-reinforcement-learning 공식 구현 pytorch

Tasks

Model Predictive Controlreinforcement-learning Reinforcement Learning

Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control

2023-11-07 · Baha Zarrouki, Chenyang Wang, Johannes Betz

In this paper, we present a Deep Reinforcement Learning (RL)-driven Adaptive Stochastic Nonlinear Model Predictive Control (SNMPC) to optimize uncertainty handling, constraints robustification, feasibility, and closed-lo…

Decision Making Deep Reinforcement Learning Model Predictive Control Reinforcement Learning (RL)

From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions

2019-12-07 · Warren B. Powell

There are over 15 distinct communities that work in the general area of sequential decisions and information, often referred to as decisions under uncertainty or stochastic optimization. We focus on two of the most impor…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Stochastic Optimization

Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning

2023-09-11 · Marin Vlastelica, Sebastian Blaes, Cristina Pineri, Georg Martius

We introduce a simple but effective method for managing risk in model-based reinforcement learning with trajectory sampling that involves probabilistic safety constraints and balancing of optimism in the face of epistemi…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

2021-07-16 · Bo Pang, Zhong-Ping Jiang

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-Free Adaptive Optimal Control of Episodic Fixed-Horizon Manufacturing Processes using Reinforcement Learning

2018-09-18 · Johannes Dornheim, Norbert Link, Peter Gumbsch

A self-learning optimal control algorithm for episodic fixed-horizon manufacturing processes with time-discrete control actions is proposed and evaluated on a simulated deep drawing process. The control model is built du…

Model Predictive Control Q-Learning reinforcement-learning Reinforcement Learning+2