Imitation Learning from Pixel Observations for Continuous Control
We study imitation learning using only visual observations for controlling dynamical systems with continuous states and actions. This setting is attractive due to the large amount of video data available from which agents could learn from. However, it is challenging due to $i)$ not observing the actions and $ii)$ the high-dimensional visual space. In this setting, we explore recipes for imitation learning based on adversarial learning and optimal transport. A key feature of our methods is to use representations from the RL encoder to compute imitation rewards. These recipes enable us to scale these methods to attain expert-level performance on visual continuous control tasks in the DeepMind control suite. We investigate the tradeoffs of these approaches and present a comprehensive evaluation of the key design choices. To encourage reproducible research in this area, we provide an easy-to-use implementation for benchmarking visual imitation learning, including our methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Benchmarkingcontinuous-controlContinuous ControlImitation LearningSimilar Papers 제목 키워드 기반
PixelBrax: Learning Continuous Control from Pixels End-to-End on the GPU
We present PixelBrax, a set of continuous control tasks with pixel observations. We combine the Brax physics engine with a pure JAX renderer, allowing reinforcement learning (RL) experiments to run end-to-end on the GPU.…
Benchmarkingcontinuous-controlContinuous ControlCPU+2Adversarial Imitation Learning from Visual Observations using Latent Information
We focus on the problem of imitation learning from visual observations, where the learning agent has access to videos of experts as its sole learning source. The challenges of this framework include the absence of expert…
Imitation LearningData-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models
Data-efficient reinforcement learning (RL) in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. We consider a particularly important i…
Model-based Reinforcement LearningModel Predictive Controlreinforcement-learningReinforcement Learning+1Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control. DrQ-v2 builds on DrQ, an off-policy actor-critic approach that uses data augmentation to learn directly from pixels. We…
continuous-controlContinuous ControlData AugmentationGPU+4From Pixels to Torques: Policy Learning with Deep Dynamical Models
Data-efficient learning in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. In this paper, we consider one instance of this challenge…
Model-based Reinforcement LearningModel Predictive Controlreinforcement-learningReinforcement Learning+1