Constrained Policy Optimization via Bayesian World Models
Improving sample-efficiency and safety are crucial challenges when deploying reinforcement learning in high-stakes real world applications. We propose LAMBDA, a novel model-based approach for policy optimization in safety critical tasks modeled via constrained Markov decision processes. Our approach utilizes Bayesian world models, and harnesses the resulting uncertainty to maximize optimistic upper bounds on the task objective, as well as pessimistic upper bounds on the safety constraints. We demonstrate LAMBDA's state of the art performance on the Safety-Gym benchmark suite in terms of sample efficiency and constraint violation.
Code (1)
Tasks
reinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Constrained Bayesian Experimental Design via Online Planning
Bayesian experimental design (BED) is a principled framework for data-efficient design of sequential experiments. However, existing BED methods are unable to adapt to dynamic constraints inherent in real-world tasks due …
Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process. In this work, we propose CERL, a new method for solving constrained Markov deci…
Safe ExplorationBayesian Optimization for Policy Search via Online-Offline Experimentation
Online field experiments are the gold-standard way of evaluating changes to real-world interactive machine learning systems. Yet our ability to explore complex, multi-dimensional policy spaces - such as those found in re…
Bayesian OptimizationBIG-bench Machine LearningConstrained Bayesian Optimization under Bivariate Gaussian Process with Application to Cure Process Optimization
Bayesian Optimization, leveraging Gaussian process models, has proven to be a powerful tool for minimizing expensive-to-evaluate objective functions by efficiently exploring the search space. Extensions such as constrain…
Bayesian OptimizationGaussian ProcessesTransition Constrained Bayesian Optimization via Markov Decision Processes
Bayesian optimization is a methodology to optimize black-box functions. Traditionally, it focuses on the setting where you can arbitrarily query the search space. However, many real-life problems do not offer this flexib…
Bayesian Optimization