Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces
Traditional controllers have limitations as they rely on prior knowledge about the physics of the problem, require modeling of dynamics, and struggle to adapt to abnormal situations. Deep reinforcement learning has the potential to address these problems by learning optimal control policies through exploration in an environment. For safety-critical environments, it is impractical to explore randomly, and replacing conventional controllers with black-box models is also undesirable. Also, it is expensive in continuous state and action spaces, unless the search space is constrained. To address these challenges we propose a specialized deep residual policy safe reinforcement learning with a cycle of learning approach adapted for complex and continuous state-action spaces. Residual policy learning allows learning a hybrid control architecture where the reinforcement learning agent acts in synchronous collaboration with the conventional controller. The cycle of learning initiates the policy through the expert trajectory and guides the exploration around it. Further, the specialization through the input-output hidden Markov model helps to optimize policy that lies within the region of interest (such as abnormality), where the reinforcement learning agent is required and is activated. The proposed solution is validated on the Tennessee Eastman process control.
Code (1)
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningSafe Reinforcement LearningSimilar Papers 제목 키워드 기반
Residual Policy Learning
We present Residual Policy Learning (RPL): a simple method for improving nondifferentiable policies using model-free deep reinforcement learning. RPL thrives in complex robotic manipulation tasks where good but imperfect…
Deep Reinforcement LearningMuJoCoreinforcement-learningReinforcement Learning+1Adaptive control of a mechatronic system using constrained residual reinforcement learning
We propose a simple, practical and intuitive approach to improve the performance of a conventional controller in uncertain environments using deep reinforcement learning while maintaining safe operation. Our approach is …
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Learning to Recover for Safe Reinforcement Learning
Safety controllers is widely used to achieve safe reinforcement learning. Most methods that apply a safety controller are using handcrafted safety constraints to construct the safety controller. However, when the environ…
reinforcement-learningReinforcement LearningRobot NavigationSafe Reinforcement LearningSafe reinforcement learning control for continuous-time nonlinear systems without a backup controller
This paper proposes an on-policy reinforcement learning (RL) control algorithm that solves the optimal regulation problem for a class of uncertain continuous-time nonlinear systems under user-defined state constraints. W…
reinforcement-learningReinforcement Learning (RL)Safe Reinforcement LearningRobust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics
This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic…
Learning Theoryreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1