DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations

2022-09-28 · Zoey Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, Dieter Fox

Teaching a multi-fingered dexterous robot to grasp objects in the real world has been a challenging problem due to its high dimensional state and action space. We propose a robot-learning system that can take a small number of human demonstrations and learn to grasp unseen object poses given partially occluded observations. Our system leverages a small motion capture dataset and generates a large dataset with diverse and successful trajectories for a multi-fingered robot gripper. By adding domain randomization, we show that our dataset provides robust grasping trajectories that can be transferred to a policy learner. We train a dexterous grasping policy that takes the point clouds of the object as input and predicts continuous actions to grasp objects from different initial robot states. We evaluate the effectiveness of our system on a 22-DoF floating Allegro Hand in simulation and a 23-DoF Allegro robot hand with a KUKA arm in real world. The policy learned from our dataset can generalize well on unseen object poses in both simulation and the real world

📄 PDF Abstract BibTeX arXiv:2209.14284

Code (0)

등록된 구현이 없습니다.

Tasks

Object

Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost

2018-10-14 · Henry Zhu, Abhishek Gupta, Aravind Rajeswaran, Sergey Levine 외

Dexterous multi-fingered robotic hands can perform a wide range of manipulation skills, making them an appealing component for general-purpose robotic manipulators. However, such hands pose a major challenge for autonomo…

Contact-rich Manipulation Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation

2023-09-06 · Zheyuan Hu, Aaron Rovinsky, Jianlan Luo, Vikash Kumar 외

Dexterous manipulation tasks involving contact-rich interactions pose a significant challenge for both model-based control systems and imitation learning algorithms. The complexity arises from the need for multi-fingered…

Imitation Learning Reinforcement Learning (RL)

Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance

2022-12-19 · Kelvin Xu, Zheyuan Hu, Ria Doshi, Aaron Rovinsky 외

Complex and contact-rich robotic manipulation tasks, particularly those that involve multi-fingered hands and underactuated object manipulation, present a significant challenge to any control method. Methods based on rei…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Do as I Do: Dexterous Manipulation Data from Everyday Human Videos

2026-06-17 · Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel 외 arxiv

How can we scalably generate data for robotic manipulation, especially on human-like platforms such as dexterous multi-fingered hands? Learning from human videos has recently emerged as a likely answer to this question. …

Robot Manipulation

Twisting Lids Off with Two Hands

2024-03-04 · Toru Lin, Zhao-Heng Yin, Haozhi Qi, Pieter Abbeel 외

Manipulating objects with two multi-fingered hands has been a long-standing challenge in robotics, due to the contact-rich nature of many manipulation tasks and the complexity inherent in coordinating a high-dimensional …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)