Learning robot motor skills with mixed reality
Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots. Related works have used MR interfaces to communicate robot intents and beliefs to a co-located human, as well as developed algorithms for taking multi-modal human input and learning complex motor behaviors. Even with these successes, enabling end-users to teach robots complex motor tasks still poses a challenge because end-user communication is highly task dependent and world knowledge is highly varied. We propose a learning framework where end-users teach robots a) motion demonstrations, b) task constraints, c) planning representations, and d) object information, all of which are integrated into a single motor skill learning framework based on Dynamic Movement Primitives (DMPs). We hypothesize that conveying this world knowledge will be intuitive with an MR interface, and that a sample-efficient motor skill learning framework which incorporates varied modalities of world knowledge will enable robots to effectively solve complex tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Mixed RealityWorld KnowledgeSimilar Papers 제목 키워드 기반
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe h…
Imitation LearningHolo-Dex: Teaching Dexterity with Immersive Mixed Reality
A fundamental challenge in teaching robots is to provide an effective interface for human teachers to demonstrate useful skills to a robot. This challenge is exacerbated in dexterous manipulation, where teaching high-dim…
Mixed RealityA Central Motor System Inspired Pre-training Reinforcement Learning for Robotic Control
The development of intelligent robots requires control policies that can handle dynamic environments and evolving tasks. Pre-training reinforcement learning has emerged as an effective approach to address these demands b…
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningExample-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks. Reinforcement learning (RL) is a promising approach for ac…
Model-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Identifying Important Sensory Feedback for Learning Locomotion Skills
Robot motor skills can be learned through deep reinforcement learning (DRL) by neural networks as state-action mappings. While the selection of state observations is crucial, there has been a lack of quantitative analysi…
Deep Reinforcement Learning