Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

2026-05-10 · Kejia Ren, Gaotian Wang, Andrew S. Morgan, Kaiyu Hang arxiv

Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.

📄 PDF Abstract BibTeX arXiv:2605.09789

Code (0)

등록된 구현이 없습니다.

Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction

2026-09-07 · Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang 외 arxiv

Human videos are an abundant source of dexterous manipulation behaviors, but they lack tactile information that is crucial for contact-rich interaction. This raises a fundamental question: can robots learn deployable vis…

Zero-shot Generalization

AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning

2026-08-14 · Zhiyue Zhao, Jingyi Wu, Hairuo Liu, Mingyu Liu 외 arxiv

Dexterous manipulation is a fundamental capability for embodied intelligence, but scaling it remains difficult because robot demonstrations are expensive to collect and action spaces vary across embodiments. Policies tra…

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

2026-07-03 · Luis Felipe Casas, Robert Teal, Keval Shah, Abhijit Tadepalli 외 arxiv

Robot manipulation policies are typically tied to specific robotic hand embodiments, limiting the transfer of learned behaviors across platforms with different kinematic structures. In this work, we propose the Unified H…

Reinforcement Learning Robot Manipulation

EgoEngine: From Egocentric Human Videos to High-Fidelity Dexterous Robot Demonstrations

2026-06-10 · Yangcen Liu, Shuo Cheng, Xinchen Yin, Woo Chul Shin 외 arxiv

Dexterous manipulation is limited by the cost of collecting large-scale robot demonstrations. Egocentric human videos offer a scalable source of diverse manipulation behaviors, but directly using them for robot learning …

DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videos

2026-02-10 · Juncheng Mu, Sizhe Yang, Yiming Bao, Hojin Bae 외 arxiv

Data scarcity fundamentally limits the generalization of bimanual dexterous manipulation, as real-world data collection for dexterous hands is expensive and labor-intensive. Human manipulation videos, as a direct carrier…

Data Augmentation Video Generation