Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills.
Code (3)
Tasks
Imitation LearningSimilar Papers 제목 키워드 기반
Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations
Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulat…
GPUImitation LearningReinforcement Learning (RL)Visual-Inertial Telepresence for Aerial Manipulation
This paper presents a novel telepresence system for enhancing aerial manipulation capabilities. It involves not only a haptic device, but also a virtual reality that provides a 3D visual feedback to a remotely-located te…
ObjectObject TrackingARMimic: Learning Robotic Manipulation from Passive Human Demonstrations in Augmented Reality
Imitation learning is a powerful paradigm for robot skill acquisition, yet conventional demonstration methods--such as kinesthetic teaching and teleoperation--are cumbersome, hardware-heavy, and disruptive to workflows. …
A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual …
PositionViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented Reality
In Augmented Reality (AR), virtual content enhances user experience by providing additional information. However, improperly positioned or designed virtual content can be detrimental to task performance, as it can impair…
Language ModelingLanguage Modelling