Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies

2017-09-18 · Fangyi Zhang, Jürgen Leitner, ZongYuan Ge, Michael Milford, Peter Corke

Various approaches have been proposed to learn visuo-motor policies for real-world robotic applications. One solution is first learning in simulation then transferring to the real world. In the transfer, most existing approaches need real-world images with labels. However, the labelling process is often expensive or even impractical in many robotic applications. In this paper, we propose an adversarial discriminative sim-to-real transfer approach to reduce the cost of labelling real data. The effectiveness of the approach is demonstrated with modular networks in a table-top object reaching task where a 7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter through visual observations. The adversarial transfer approach reduced the labelled real data requirement by 50%. Policies can be transferred to real environments with only 93 labelled and 186 unlabelled real images. The transferred visuo-motor policies are robust to novel (not seen in training) objects in clutter and even a moving target, achieving a 97.8% success rate and 1.8 cm control accuracy.

📄 PDF Abstract BibTeX arXiv:1709.05746

Code (1)

Fanleyrobot/ADT 공식 구현 torch

Adversarial Feature Training for Generalizable Robotic Visuomotor Control

2019-09-17 · Xi Chen, Ali Ghadirzadeh, Mårten Björkman, Patric Jensfelt

Deep reinforcement learning (RL) has enabled training action-selection policies, end-to-end, by learning a function which maps image pixels to action outputs. However, it's application to visuomotor robotic policy traini…

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Beyond the Patch: Exploring Vulnerabilities of Visuomotor Policies via Viewpoint-Consistent 3D Adversarial Object

2026-03-05 · Chanmi Lee, Minsung Yoon, Woojae Kim, Sebin Lee 외 arxiv

Neural network-based visuomotor policies enable robots to perform manipulation tasks but remain susceptible to perceptual attacks. For example, conventional 2D adversarial patches are effective under fixed-camera setups,…

Visuomotor Grasping with World Models for Surgical Robots

2025-08-15 · Hongbin Lin, Bin Li, Kwok Wai Samuel Au arxiv

Grasping is a fundamental task in robot-assisted surgery (RAS), and automating it can reduce surgeon workload while enhancing efficiency, safety, and consistency beyond teleoperated systems. Most prior approaches rely on…

Pose Tracking

Reinforcement and Imitation Learning for Diverse Visuomotor Skills

2018-02-26 · ICLR 2018 1 · Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu 외

We propose a model-free deep reinforcement learning method that leverages a small amount of demonstration data to assist a reinforcement learning agent. We apply this approach to robotic manipulation tasks and train end-…

Deep Reinforcement Learning Imitation Learning reinforcement-learning Reinforcement Learning+1

CHOP: Counterfactual Human Preference Labels Improve Obstacle Avoidance in Visuomotor Navigation Policies

2026-03-02 · Gershom Seneviratne, Jianyu An, Vaibhav Shende, Sahire Ellahy 외 arxiv

Visuomotor navigation policies have shown strong perception-action coupling for embodied agents, yet they often struggle with safe navigation and dynamic obstacle avoidance in complex real-world environments. We introduc…