Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning
We consider the problem of detecting out-of-distribution (OOD) samples in deep reinforcement learning. In a value based reinforcement learning setting, we propose to use uncertainty estimation techniques directly on the agent's value estimating neural network to detect OOD samples. The focus of our work lies in analyzing the suitability of approximate Bayesian inference methods and related ensembling techniques that generate uncertainty estimates. Although prior work has shown that dropout-based variational inference techniques and bootstrap-based approaches can be used to model epistemic uncertainty, the suitability for detecting OOD samples in deep reinforcement learning remains an open question. Our results show that uncertainty estimation can be used to differentiate in- from out-of-distribution samples. Over the complete training process of the reinforcement learning agents, bootstrap-based approaches tend to produce more reliable epistemic uncertainty estimates, when compared to dropout-based approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceDeep Reinforcement LearningOpen-Ended Question AnsweringOut-of-Distribution Detectionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Variational InferenceSimilar Papers 제목 키워드 기반
Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning
Reinforcement Learning (RL) based solutions are being adopted in a variety of domains including robotics, health care and industrial automation. Most focus is given to when these solutions work well, but they fail when p…
Deep Reinforcement LearningOut-of-Distribution DetectionOut of Distribution (OOD) Detectionreinforcement-learning+2Safe Domain Randomization via Uncertainty-Aware Out-of-Distribution Detection and Policy Adaptation
Deploying reinforcement learning (RL) policies in real-world involves significant challenges, including distribution shifts, safety concerns, and the impracticality of direct interactions during policy refinement. Existi…
MuJoCoOut-of-Distribution DetectionOut of Distribution (OOD) DetectionReinforcement Learning (RL)Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs
Uncertainty quantification is crucial for building reliable and trustable machine learning systems. We propose to estimate uncertainty in recurrent neural networks (RNNs) via stochastic discrete state transitions over re…
Out-of-Distribution Detectionreinforcement-learningReinforcement Learning (RL)Uncertainty QuantificationHow to Enable Uncertainty Estimation in Proximal Policy Optimization
While deep reinforcement learning (RL) agents have showcased strong results across many domains, a major concern is their inherent opaqueness and the safety of such systems in real-world use cases. To overcome these issu…
Deep Reinforcement LearningOut of Distribution (OOD) DetectionReinforcement Learning (RL)Learning to Navigate Under Imperfect Perception: Conformalised Segmentation for Safe Reinforcement Learning
Reliable navigation in safety-critical environments requires both accurate hazard perception and principled uncertainty handling to strengthen downstream safety handling. Despite the effectiveness of existing approaches,…
Reinforcement LearningSemantic Segmentation