Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process. In this work, we propose CERL, a new method for solving constrained Markov decision processes while keeping the policy safe during learning. Our method leverages Bayesian world models and suggests policies that are pessimistic w.r.t. the model's epistemic uncertainty. This makes CERL robust towards model inaccuracies and leads to safe exploration during learning. In our experiments, we demonstrate that CERL outperforms the current state-of-the-art in terms of safety and optimality in solving CMDPs from image observations.
Code (0)
등록된 구현이 없습니다.
Tasks
Safe ExplorationSimilar Papers 제목 키워드 기반
Kernel-Based Safe Exploration in Deep Reinforcement Learning
Safety has been a major concern when deploying deep reinforcement learning algorithms in the real world. A promising direction that ensures that the learned policy does not visit unsafe regions is to learn a \emph{barrie…
Reinforcement LearningContinuous ControlSafe Bayesian Optimization for the Control of High-Dimensional Embodied Systems
Learning to move is a primary goal for animals and robots, where ensuring safety is often important when optimizing control policies on the embodied systems. For complex tasks such as the control of human or humanoid con…
Bayesian OptimizationHumanoid ControlSafe ExplorationSafe Importance Sampling in Model Predictive Path Integral Control
We introduce the notion of importance sampling under embedded barrier state control, titled Safety Controlled Model Predictive Path Integral Control (SC-MPPI). For robotic systems operating in an environment with multipl…
Optimal Parameter Adaptation for Safety-Critical Control via Safe Barrier Bayesian Optimization
Safety is of paramount importance in control systems to avoid costly risks and catastrophic damages. The control barrier function (CBF) method, a promising solution for safety-critical control, poses a new challenge of e…
Bayesian OptimizationSafe Exploration for Interactive Machine Learning
In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in appli…
Active LearningBayesian OptimizationBIG-bench Machine LearningSafe Exploration