IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic
This paper introduces the SOAR framework for imitation learning. SOAR is an algorithmic template that learns a policy from expert demonstrations with a primal dual style algorithm that alternates cost and policy updates. Within the policy updates, the SOAR framework uses an actor critic method with multiple critics to estimate the critic uncertainty and build an optimistic critic fundamental to drive exploration. When instantiated in the tabular setting, we get a provable algorithm with guarantees that matches the best known results in $\epsilon$. Practically, the SOAR template is shown to boost consistently the performance of imitation learning algorithms based on Soft Actor Critic such as f-IRL, ML-IRL and CSIL in several MuJoCo environments. Overall, thanks to SOAR, the required number of episodes to achieve the same performance is reduced by half.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningMuJoCoMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Wasserstein Barycenter Soft Actor-Critic
Deep off-policy actor-critic algorithms have emerged as the leading framework for reinforcement learning in continuous control domains. However, most of these algorithms suffer from poor sample efficiency, especially in …
continuous-controlContinuous ControlMuJoCoOptimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
Although actor-critic methods have been successful in practice, their theoretical analyses have several limitations. Specifically, existing theoretical work either sidesteps the exploration problem by making strong assum…
Better Exploration with Optimistic Actor-Critic
Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of …
continuous-controlContinuous ControlEfficient ExplorationReinforcement LearningBetter Exploration with Optimistic Actor Critic
Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of …
continuous-controlContinuous ControlEfficient ExplorationReinforcement LearningSoar: Design and Deployment of A Smart Roadside Infrastructure System for Autonomous Driving
Recently,smart roadside infrastructure (SRI) has demonstrated the potential of achieving fully autonomous driving systems. To explore the potential of infrastructure-assisted autonomous driving, this paper presents the d…
Autonomous Driving