Iterative Batch Reinforcement Learning via Safe Diversified Model-based Policy Search
Batch reinforcement learning enables policy learning without direct interaction with the environment during training, relying exclusively on previously collected sets of interactions. This approach is, therefore, well-suited for high-risk and cost-intensive applications, such as industrial control. Learned policies are commonly restricted to act in a similar fashion as observed in the batch. In a real-world scenario, learned policies are deployed in the industrial system, inevitably leading to the collection of new data that can subsequently be added to the existing recording. The process of learning and deployment can thus take place multiple times throughout the lifespan of a system. In this work, we propose to exploit this iterative nature of applying offline reinforcement learning to guide learned policies towards efficient and informative data collection during deployment, leading to continuous improvement of learned policies while remaining within the support of collected data. We present an algorithmic methodology for iterative batch reinforcement learning based on ensemble-based model-based policy search, augmented with safety and, importantly, a diversity criterion.
Code (0)
등록된 구현이 없습니다.
Tasks
Diversityreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Iterative Learning Control-Informed Reinforcement Learning for Batch Process Control
A significant limitation of Deep Reinforcement Learning (DRL) is the stochastic uncertainty in actions generated during exploration-exploitation, which poses substantial safety risks during both training and deployment. …
Reinforcement LearningKFCPO: Kronecker-Factored Approximated Constrained Policy Optimization
We propose KFCPO, a novel Safe Reinforcement Learning (Safe RL) algorithm that combines scalable Kronecker-Factored Approximate Curvature (K-FAC) based second-order policy optimization with safety-aware gradient manipula…
Reinforcement LearningConstrained Policy Improvement for Safe and Efficient Reinforcement Learning
We propose a policy improvement algorithm for Reinforcement Learning (RL) which is called Rerouted Behavior Improvement (RBI). RBI is designed to take into account the evaluation errors of the Q-function. Such errors are…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Distributed Policy Learning Based Random Access for Diversified QoS Requirements
Future wireless access networks need to support diversified quality of service (QoS) metrics required by various types of Internet-of-Things (IoT) devices, e.g., age of information (AoI) for status generating sources and…
Multi-batch Reinforcement Learning via Sample Transfer and Imitation Learning
Reinforcement learning (RL), especially deep reinforcement learning, has achieved impressive performance on different control tasks. Unfortunately, most online reinforcement learning algorithms require a large number of …
continuous-controlContinuous ControlDeep Reinforcement LearningImitation Learning+4