Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation

2022-06-06 · YiWen Chen, Xue Li, Sheng Guo, Xian Yao Ng, Marcelo Ang

Reinforcement learning has shown a wide usage in robotics tasks, such as insertion and grasping. However, without a practical sim2real strategy, the policy trained in simulation could fail on the real task. There are also wide researches in the sim2real strategies, but most of those methods rely on heavy image rendering, domain randomization training, or tuning. In this work, we solve the insertion task using a pure visual reinforcement learning solution with minimum infrastructure requirement. We also propose a novel sim2real strategy, Real2Sim, which provides a novel and easier solution in policy adaptation. We discuss the advantage of Real2Sim compared with Sim2Real.

📄 PDF Abstract BibTeX arXiv:2206.02679

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera

2020-05-29 · Damian Bogunowicz, Aleksandr Rybnikov, Komal Vendidandi, Fedor Chervinskii

Even though the peg-hole insertion is one of the well-studied problems in robotics, it still remains a challenge for robots, especially when it comes to flexibility and the ability to generalize. Successful completion of…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

KOVIS: Keypoint-based Visual Servoing with Zero-Shot Sim-to-Real Transfer for Robotics Manipulation

2020-07-28 · En Yen Puang, Keng Peng Tee, Wei Jing

We present KOVIS, a novel learning-based, calibration-free visual servoing method for fine robotic manipulation tasks with eye-in-hand stereo camera system. We train the deep neural network only in the simulated environm…

Data Augmentation Self-Supervised Learning

Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards

2019-06-13 · Gerrit Schoettler, Ashvin Nair, Jianlan Luo, Shikhar Bahl 외

Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling,…

Deep Reinforcement Learning Friction Reinforcement Learning Reinforcement Learning (RL)

Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

2017-07-27 · Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang 외

We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards. We build upon the Deep Deterministic Policy Gradient (DDPG) algorithm to use demonstrations. Both demonst…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Touch2Insert: Zero-Shot Peg Insertion by Touching Intersections of Peg and Hole

2026-03-04 · Masaru Yajima, Yuma Shin, Rei Kawakami, Asako Kanezaki 외 arxiv

Reliable insertion of industrial connectors remains a central challenge in robotics, requiring sub-millimeter precision under uncertainty and often without full visual access. Vision-based approaches struggle with occlus…

Pose Estimation