WinTok: A Win-Win Hybrid Tokenizer via Decomposing Visual Understanding and Generation with Transferable Tokens

2026-05-18 · Yiwei Guo, Shaobin Zhuang, Zhipeng Huang, Canmiao Fu, Chen Li, Jing Lyu, Yali Wang arxiv

Building a unified visual tokenizer is essential for bridging the gap between visual understanding and generation. Yet existing approaches struggle with the inherent conflict between these tasks, as a single token space is forced to support both high-level semantic abstraction and low-level pixel reconstruction. We propose WinTok, a concise hybrid tokenizer that achieves a win-win performance by explicitly decoupling the two objectives. WinTok supplements pixel tokens with a set of learnable semantic tokens, effectively mitigating cross-task interference without incurring the computational overhead of dual tokenizers. To further enhance understanding capability, we introduce an asymmetric token distillation mechanism: the semantic tokens are guided by pretrained semantic embeddings from any visual foundation model, enabling them to inherit strong discriminative power while maintaining flexibility. Across 10 challenging benchmarks, WinTok delivers consistent improvements in reconstruction, understanding, and generation. Trained on only 50M open-source data, WinTok surpasses the strong baseline UniTok by 11.2% in classification accuracy and achieves a competitive reconstruction rFID of 0.41, despite using substantially less training data. Code is released at https://github.com/markywg/WinTok.

📄 PDF Abstract BibTeX arXiv:2605.18115

Code (0)

등록된 구현이 없습니다.

MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer

2025-09-19 · Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang 외 arxiv

Unified multimodal Large Language Models (LLMs) that can both understand and generate visual content hold immense potential. However, existing open-source models often suffer from a performance trade-off between these ca…

Text-to-Image Generation

Factorized Visual Tokenization and Generation

2024-11-25 · Zechen Bai, Jianxiong Gao, Ziteng Gao, Pichao Wang 외

Visual tokenizers are fundamental to image generation. They convert visual data into discrete tokens, enabling transformer-based models to excel at image generation. Despite their success, VQ-based tokenizers like VQGAN …

Disentanglement Image Generation Quantization Representation Learning

What Makes for Good Visual Tokenizers for Large Language Models?

2023-05-20 · Guangzhi Wang, Yixiao Ge, Xiaohan Ding, Mohan Kankanhalli 외

We empirically investigate proper pre-training methods to build good visual tokenizers, making Large Language Models (LLMs) powerful Multimodal Large Language Models (MLLMs). In our benchmark, which is curated to evaluat…

Image Captioning Object Counting Question Answering Visual Question Answering

UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation

2025-10-12 · Zhengrong Yue, Haiyu Zhang, Xiangyu Zeng, Boyu Chen 외 arxiv

Tokenizer is a crucial component for both visual understanding and generation. To advance toward the ultimate goal of universal modeling, recent research has focused on developing a unified tokenizer. However, existing t…

HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

2024-10-14 · Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie 외

We introduce Hybrid Autoregressive Transformer (HART), an autoregressive (AR) visual generation model capable of directly generating 1024x1024 images, rivaling diffusion models in image generation quality. Existing AR mo…

Image Generation Image Reconstruction