SigLIP-HD by Fine-to-Coarse Supervision
High-quality visual representation is a long-standing pursuit in computer vision. In the context of multimodal LLMs (MLLMs), feeding higher-resolution images can produce more fine-grained visual tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post-processing of increased tokens. Before simply adopting a higher resolution, have we truly unlocked the model's full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present SigLIP-HD in this work. The core is a highly simple fine-to-coarse supervision design. We enforce the coarse feature of a mid-resolution image to mimic the fine-grained feature of its high-resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual tokens at exactly the same inference budget. It is validated on extensive MLLM benchmarks and consistently delivers stronger results than our baseline model, especially on OCR-related tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CT
Recent 3D CT vision-language models align volumes with reports via contrastive pretraining, but typically rely on limited public data and provide only coarse global supervision. We train a 3D CT vision-language model on …
Image RetrievalFine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning
In this paper, we propose a new task named Fine-grained Category Discovery under Coarse-grained supervision (FCDC). Without asking for any fine-grained knowledge, FCDC aims at discovering fine-grained categories with onl…
Contrastive LearningZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval
Adapting a foundation vision-language encoder to a specialized retrieval task creates a fundamental tradeoff: gains on the target distribution come at the cost of the foundation model's broad generalization, and fashion …
Knowledge DistillationFineViT: Progressively Unlocking Fine-Grained Perception with Dense Recaptions
While Multimodal Large Language Models (MLLMs) have experienced rapid advancements, their visual encoders frequently remain a performance bottleneck. Conventional CLIP-based encoders struggle with dense spatial tasks due…
LucidFlux: Caption-Free Photo-Realistic Image Restoration via a Large-Scale Diffusion Transformer
Image restoration (IR) aims to recover images degraded by unknown mixtures while preserving semanticsconditions under which discriminative restorers and UNet-based diffusion priors often oversmooth, hallucinate, or drift…
Image Restoration