Soft-Robust Algorithms for Batch Reinforcement Learning
In reinforcement learning, robust policies for high-stakes decision-making problems with limited data are usually computed by optimizing the percentile criterion, which minimizes the probability of a catastrophic failure. Unfortunately, such policies are typically overly conservative as the percentile criterion is non-convex, difficult to optimize, and ignores the mean performance. To overcome these shortcomings, we study the soft-robust criterion, which uses risk measures to balance the mean and percentile criterion better. In this paper, we establish the soft-robust criterion's fundamental properties, show that it is NP-hard to optimize, and propose and analyze two algorithms to approximately optimize it. Our theoretical analyses and empirical evaluations demonstrate that our algorithms compute much less conservative solutions than the existing approximate methods for optimizing the percentile-criterion.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Towards Batch-to-Streaming Deep Reinforcement Learning for Continuous Control
State-of-the-art deep reinforcement learning (RL) methods have achieved remarkable performance in continuous control tasks, yet their computational complexity is often incompatible with the constraints of resource-limite…
Reinforcement LearningContinuous ControlBenchmarking Batch Deep Reinforcement Learning Algorithms
Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment. Following this result, there have been several pap…
BenchmarkingDeep Reinforcement LearningQ-Learningreinforcement-learning+2Multi-batch Reinforcement Learning via Sample Transfer and Imitation Learning
Reinforcement learning (RL), especially deep reinforcement learning, has achieved impressive performance on different control tasks. Unfortunately, most online reinforcement learning algorithms require a large number of …
continuous-controlContinuous ControlDeep Reinforcement LearningImitation Learning+4Multi-timestep models for Model-based Reinforcement Learning
In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories from one-step dynamics models learned on data. A critical challenge of this approach is the compounding of one-step prediction…
modelModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1Predicting the Impact of Batch Refactoring Code Smells on Application Resource Consumption
Automated batch refactoring has become a de-facto mechanism to restructure software that may have significant design flaws negatively impacting the code quality and maintainability. Although automated batch refactoring t…
CPU