Feeling the Unexpected: ResTacVLA for Contact-Rich Manipulation via Residual Tactile Representation
Tactile perception is indispensable for contact-rich manipulation, yet integrating it into Vision-Language-Action (VLA) models often induces modality collapse, where high-bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high-value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact-rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: https://awilekong.github.io/ResTacVLA/
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Contact-Safe Reinforcement Learning Framework for Contact-Rich Robot Manipulation
Reinforcement learning shows great potential to solve complex contact-rich robot manipulation tasks. However, the safety of using RL in the real world is a crucial problem, since unexpected dangerous collisions might hap…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot Manipulation+1Learning Compliance Adaptation in Contact-Rich Manipulation
Compliant robot behavior is crucial for the realization of contact-rich manipulation tasks. In such tasks, it is important to ensure a high stiffness and force tracking accuracy during normal task execution as well as ra…
Anomaly DetectionContact-rich ManipulationZero-Shot Transfer of Haptics-Based Object Insertion Policies
Humans naturally exploit haptic feedback during contact-rich tasks like loading a dishwasher or stocking a bookshelf. Current robotic systems focus on avoiding unexpected contact, often relying on strategically placed en…
A Soft Wrist with Anisotropic and Selectable Stiffness for Robust Robot Learning in Contact-rich Manipulation
Contact-rich manipulation tasks in unstructured environments pose significant robustness challenges for robot learning, where unexpected collisions can cause damage and hinder policy acquisition. Existing soft end-effect…
Uncertainty-aware Contact-safe Model-based Reinforcement Learning
This letter presents contact-safe Model-based Reinforcement Learning (MBRL) for robot applications that achieves contact-safe behaviors in the learning process. In typical MBRL, we cannot expect the data-driven model to …
Contact-rich ManipulationModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1