OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
While deep reinforcement learning techniques have recently produced considerable achievements on many decision-making problems, their use in robotics has largely been limited to simulated worlds or restricted motions, since unconstrained trial-and-error interactions in the real world can have undesirable consequences for the robot or its environment. To overcome such limitations, we propose a novel reinforcement learning architecture, OptLayer, that takes as inputs possibly unsafe actions predicted by a neural network and outputs the closest actions that satisfy chosen constraints. While learning control policies often requires carefully crafted rewards and penalties while exploring the range of possible actions, OptLayer ensures that only safe actions are actually executed and unsafe predictions are penalized during training. We demonstrate the effectiveness of our approach on robot reaching tasks, both simulated and in the real world.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systems
Safe reinforcement learning (RL) with hard constraint guarantees is a promising optimal control direction for multi-energy management systems. It only requires the environment-specific constraint functions itself a prior…
energy managementManagementReinforcement Learning (RL)Safe Reinforcement LearningReal-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems
Recent advancements in machine learning based energy management approaches, specifically reinforcement learning with a safety layer (OptLayerPolicy) and a metaheuristic algorithm generating a decision tree control policy…
energy managementManagementModel Predictive Controlreinforcement-learning+2SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization
Incorporating safety is an essential prerequisite for broadening the practical applications of reinforcement learning in real-world scenarios. To tackle this challenge, Constrained Markov Decision Processes (CMDPs) are l…
Benchmarkingreinforcement-learningReinforcement LearningSafe Reinforcement LearningConstrained Meta Agnostic Reinforcement Learning
Meta-Reinforcement Learning (Meta-RL) aims to acquire meta-knowledge for quick adaptation to diverse tasks. However, applying these policies in real-world environments presents a significant challenge in balancing rapid …
Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement LearningModel-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithm
During initial iterations of training in most Reinforcement Learning (RL) algorithms, agents perform a significant number of random exploratory steps. In the real world, this can limit the practicality of these algorithm…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Safe Exploration+1