Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation

2021-06-16 · Alex Church, John Lloyd, Raia Hadsell, Nathan F. Lepora

Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A simple and fast method of simulating optical tactile sensors is provided, where high-resolution contact geometry is represented as depth images. Proximal Policy Optimisation (PPO) is used to learn successful policies across all considered tasks. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zero-shot sim-to-real policy transfer on several physically-interactive tasks requiring a sense of touch.

📄 PDF Abstract BibTeX arXiv:2106.08796

Code (2)

ac-93/tactile_gym 공식 구현

ac-93/tactile_gym_servo_control pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Translation

VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning

2025-10-16 · Binghao Huang, Jie Xu, Iretiayo Akinola, Wei Yang 외 arxiv

Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to the suboptimality and limited diversity …

Reinforcement Learning

OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer

2025-12-09 · Jessica Yin, Haozhi Qi, Youngsun Wi, Sayantan Kundu 외 arxiv

Human video demonstrations provide abundant training data for learning robot policies, but video alone cannot capture the rich contact signals critical for mastering manipulation. We introduce OSMO, an open-source wearab…

Image Inpainting

Learning In-Hand Translation Using Tactile Skin With Shear and Normal Force Sensing

2024-07-10 · Jessica Yin, Haozhi Qi, Jitendra Malik, James Pikul 외

Recent progress in reinforcement learning (RL) and tactile sensing has significantly advanced dexterous manipulation. However, these methods often utilize simplified tactile signals due to the gap between tactile simulat…

Reinforcement Learning (RL)

TacCoRL: Integrating Tactile Feedback into VLA via Simulation

2026-06-10 · Siyu Ma, Yuqi Liang, Chang Yu, Yunuo Chen 외 arxiv

Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation, but visual observations alone often miss the local contact state required for contact-rich tasks. We present …

Reinforcement Learning Robot Manipulation

Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies

2026-04-29 · Pokuang Zhou, Yuhao Zhou, Quan Khanh Luu, Seungho Han 외 arxiv

Quadrupedal loco-manipulation is commonly built on visual perception and proprioception. Yet reliable contact-rich manipulation remains difficult: vision and proprioception alone cannot resolve uncertain, evolving intera…