paper-with-me

홈 › Papers

TacCoRL: Integrating Tactile Feedback into VLA via Simulation

2026-06-10 · Siyu Ma, Yuqi Liang, Chang Yu, Yunuo Chen, Hao Su, Yixin Zhu, Yin Yang, Chenfanfu Jiang arxiv

Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation, but visual observations alone often miss the local contact state required for contact-rich tasks. We present TacCoRL, a scalable framework that injects Tactile feedback into VLA policies and improves them through sim-real Co-training and simulation-based reinforcement learning (RL), without requiring large-scale tactile pretraining or extensive real-world contact exploration. The key idea is not only adding touch as an input, but learning how contact readings should modulate action responses in near-failure states that are rare in demonstrations and risky to collect on hardware. We use a real-aligned simulator as a closed-loop training environment for contact interaction. Mixed simulated and real trajectories first warm-start tactile-conditioned actions in the pretrained policy. Reinforcement learning with verifiable task rewards then optimizes the policy using simulated contact rollouts. It reinforces tactile-conditioned actions that lead to task completion, while a supervised objective on real trajectories keeps the refined policy anchored to deployment visual, tactile, and action distributions. The resulting policy transfers directly to the real robot without privileged simulation state or online real-world RL. Across four bimanual contact-rich tasks, the final visuo-tactile policy achieves an average success rate of 72.5%, compared to baseline of 50.0%. Result videos and more details are available at https://tac-corl.github.io/

📄 PDF Abstract BibTeX arXiv:2606.11743

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement LearningRobot Manipulation

Similar Papers 제목 키워드 기반

Combining Vision and Tactile Sensation for Video Prediction

2023-04-21 · Willow Mandil, Amir Ghalamzan-E

In this paper, we explore the impact of adding tactile sensation to video prediction models for physical robot interactions. Predicting the impact of robotic actions on the environment is a fundamental challenge in robot…

PredictionVideo Prediction

ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction

2026-08-03 · Shiqi Zhang, Xin Zhang, Yedong Shen, Yao Li 외 arxiv

Fusing tactile signals has proven effective for contact-rich manipulation, enabling robots to perceive contact states and adapt to rapidly changing physical interactions. Yet effectively integrating tactile feedback into…

TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models

2026-06-27 · Mark Van der Merwe, Mohamad Louai Shehab, Jayjun Lee, Youngsun Wi 외 arxiv

Vision-Language-Action (VLA) models demonstrate impressive reasoning over visual, semantic, and spatial task variations by leveraging large-scale vision and language pre-training. They remain, however, largely blind to c…

Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigms

2026-05-17 · Zhixiang Cao, Di Tian, Runwei Guan, Yanzhou Mu 외 arxiv

Tactile sensing is a fundamental modality for embodied intelligence, offering unique and direct feedback on contact geometry, material properties, and interaction dynamics that remote sensors cannot replace. However, uni…

ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation

2026-04-22 · Zhe Xu, Feiyu Zhao, Xiyan Huang, Chenxi Xiao arxiv

Tactile sensors are increasingly integrated into dexterous robotic manipulators to enhance contact perception. However, learning manipulation policies that rely on tactile sensing remains challenging, primarily due to th…

Reinforcement Learning