Sample-Efficient and Safe Deep Reinforcement Learning via Reset Deep Ensemble Agents

2023-10-31 · NeurIPS 2023 11

Deep reinforcement learning (RL) has achieved remarkable success in solving complex tasks through its integration with deep neural networks (DNNs) as function approximators. However, the reliance on DNNs has introduced a new challenge called primacy bias, whereby these function approximators tend to prioritize early experiences, leading to overfitting. To mitigate this primacy bias, a reset method has been proposed, which performs periodic resets of a portion or the entirety of a deep RL agent while preserving the replay buffer. However, the use of the reset method can result in performance collapses after executing the reset, which can be detrimental from the perspective of safe RL and regret minimization. In this paper, we propose a new reset-based method that leverages deep ensemble learning to address the limitations of the vanilla reset method and enhance sample efficiency. The proposed method is evaluated through various experiments including those in the domain of safe RL. Numerical results show its effectiveness in high sample efficiency and safety considerations.

📄 PDF Abstract BibTeX arXiv:2310.20287

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningEnsemble Learning Reinforcement Learning (RL)

Safe Reinforcement Learning via Curriculum Induction

2020-06-22 · NeurIPS 2020 12 · Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause 외

In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. In such settings, the agent needs to behave safely not only after but also while learning. To achi…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning

2017-11-18 · ICLR 2018 1 · Benjamin Eysenbach, Shixiang Gu, Julian Ibarz, Sergey Levine

Deep reinforcement learning algorithms can learn complex behavioral skills, but real-world application of these methods requires a large amount of experience to be collected by the agent. In practical settings, such as r…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sampling-Based Safe Reinforcement Learning

2026-05-19 · Luca Vignola, Bruce D. Lee, Manish Prajapat, Manuel Wendl 외 arxiv

Safe exploration remains a fundamental challenge in reinforcement learning (RL), limiting the deployment of RL agents in the real world. We propose Sampling-Based Safe Reinforcement Learning (SBSRL), a model-based RL alg…

Reinforcement Learning Continuous Control

Learning to reset in target search problems

2025-03-14 · Gorka Muñoz-Gil, Hans J. Briegel, Michele Caraglio

Target search problems are central to a wide range of fields, from biological foraging to the optimization algorithms. Recently, the ability to reset the search has been shown to significantly improve the searcher's effi…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Intelligent Switching for Reset-Free RL

2024-05-02 · Darshan Patil, Janarthanan Rajendran, Glen Berseth, Sarath Chandar

In the real world, the strong episode resetting mechanisms that are needed to train agents in simulation are unavailable. The \textit{resetting} assumption limits the potential of reinforcement learning in the real world…