홈 › Papers

FAST: Efficient Action Tokenization for Vision-Language-Action Models

2025-01-16 · Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, Sergey Levine

Autoregressive sequence models, such as Transformer-based vision-language action (VLA) policies, can be tremendously effective for capturing complex and generalizable robotic behaviors. However, such models require us to choose a tokenization of our continuous action signals, which determines how the discrete symbols predicted by the model map to continuous robot actions. We find that current approaches for robot action tokenization, based on simple per-dimension, per-timestep binning schemes, typically perform poorly when learning dexterous skills from high-frequency robot data. To address this challenge, we propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform. Our tokenization approach, Frequency-space Action Sequence Tokenization (FAST), enables us to train autoregressive VLAs for highly dexterous and high-frequency tasks where standard discretization methods fail completely. Based on FAST, we release FAST+, a universal robot action tokenizer, trained on 1M real robot action trajectories. It can be used as a black-box tokenizer for a wide range of robot action sequences, with diverse action spaces and control frequencies. Finally, we show that, when combined with the pi0 VLA, our method can scale to training on 10k hours of robot data and match the performance of diffusion VLAs, while reducing training time by up to 5x.

📄 PDF Abstract BibTeX arXiv:2501.09747

Code (0)

등록된 구현이 없습니다.

Tasks

Vision-Language-Action

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization

2025-12-04 · Yicheng Liu, Shiduo Zhang, Zibin Dong, Baijun Ye 외 arxiv

Autoregressive vision-language-action (VLA) models have recently demonstrated strong capabilities in robotic manipulation. However, their core process of action tokenization often involves a trade-off between reconstruct…

Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models

2025-09-28 · Rokas Bendikas, Daniel Dijkman, Markus Peschl, Sanjay Haresh 외 arxiv

Vision-Language-Action (VLA) models offer a pivotal approach to learning robotic manipulation at scale by repurposing large pre-trained Vision-Language-Models (VLM) to output robotic actions. However, adapting VLMs for r…

Representation Learning

X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining

2026-06-07 · Miracle Kang, Lights Shi, Lucy Liang, Roy Gan 외 arxiv

Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing code…

Multimodal Reasoning

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

2026-09-16 · Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin 외 hf

Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted tokens. Their fidelity is…