DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
While imitation learning is becoming common practice in robotics, this approach often suffers from data mismatch and compounding errors. DAgger is an iterative algorithm that addresses these issues by continually aggregating training data from both the expert and novice policies, but does not consider the impact of safety. We present a probabilistic extension to DAgger, which uses the distribution over actions provided by the novice policy, for a given observation. Our method, which we call DropoutDAgger, uses dropout to train the novice as a Bayesian neural network that provides insight to its confidence. Using the distribution over the novice's actions, we estimate a probabilistic measure of safety with respect to the expert action, tuned to balance exploration and exploitation. The utility of this approach is evaluated on the MuJoCo HalfCheetah and in a simple driving experiment, demonstrating improved performance and safety compared to other DAgger variants and classic imitation learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningMuJoCoMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
Bayesian reward learning from demonstrations enables rigorous safety and uncertainty analysis when performing imitation learning. However, Bayesian reward learning methods are typically computationally intractable for co…
Atari GamesBayesian InferenceImitation LearningDeep Bayesian Reward Learning from Preferences
Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy. However, Bayesian IRL is …
Atari GamesImitation LearningReinforcement LearningTuning Particle Accelerators with Safety Constraints using Bayesian Optimization
Tuning machine parameters of particle accelerators is a repetitive and time-consuming task that is challenging to automate. While many off-the-shelf optimization algorithms are available, in practice their use is limited…
Bayesian Optimizationglobal-optimizationSafety-Aware Robot Damage Recovery Using Constrained Bayesian Optimization and Simulated Priors
The recently introduced Intelligent Trial-and-Error (IT&E) algorithm showed that robots can adapt to damage in a matter of a few trials. The success of this algorithm relies on two components: prior knowledge acquired th…
Bayesian OptimizationBayesian Deep Learning for Segmentation for Autonomous Safe Planetary Landing
Hazard detection is critical for enabling autonomous landing on planetary surfaces. Current state-of-the-art methods leverage traditional computer vision approaches to automate the identification of safe terrain from inp…
Deep LearningSemantic Segmentation