Training-Free Interaction-Aligned Visual Token Pruning for Efficient Embodied Manipulation
Efficient visual representation is a central image-processing challenge in embodied manipulation, where policies repeatedly process dense visual-token sequences during closed-loop control. Existing methods rank or prune tokens using semantic relevance, VLM attention, cross-frame redundancy, or motion in the action space. These signals may discard task-relevant regions when instruction-related appearance and observed image motion are not yet spatially aligned. We introduce Interaction-Aligned Pruning (IAprune), a training-free method that treats per-frame budget setting and within-budget token selection as two linked decisions. Semantic--motion spatial agreement guides the choice between Conservative and Aggressive coverage, and the resulting region size is mapped to a calibrated dynamic budget. Continuous semantic and motion responses rank the existing tokens, while geometric residual correction redirects fixed selection slots toward under-represented boundaries without increasing the sequence length. Across four embodied manipulation policies, three simulation benchmarks, and a real-robot platform, IAprune provides a favorable accuracy--efficiency trade-off, matching the unpruned policy on LIBERO with a \(1.54\times\) speedup and reaching \(1.48\times\) acceleration on a real robot. Phase-wise analysis shows that retention gains are largest under tight budgets early in an episode, while fixed-budget analysis confirms that geometry replaces low-priority tokens with boundary and contact-region evidence rather than retaining more tokens. Our project website is: \href{https://chengjt1999.github.io/VLA-IAP.github.io/}{IAprune.com}.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards
Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training has emerged as a promising direction to…
Reinforcement LearningVideo GenerationTextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models
Image-text models excel at image-level tasks but struggle with detailed visual understanding. While these models provide strong visual-language alignment, segmentation models like SAM2 offer precise spatial boundaries fo…
Referring ExpressionReferring Expression ComprehensionSemantic SegmentationUnsupervised Semantic Segmentation with Language-image Pre-trainingEnhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding
Recent advances in Multimodal Large Language Models (MLLMs) have significantly advanced video understanding tasks, yet challenges remain in efficiently compressing visual tokens while preserving spatiotemporal interactio…
Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent
The advancement of Multimodal Large Language Models (MLLMs) has expanded Role-Playing Agents (RPAs) into visually grounded environments. However, human vision is inherently subjective and identity-driven, whereas existin…
Visual GroundingVLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising training-free strategy that avoids redu…