SPoT: Subpixel Placement of Tokens in Vision Transformers
Vision Transformers naturally accommodate sparsity, yet standard tokenization methods confine features to discrete patch grids. This constraint prevents models from fully exploiting sparse regimes, forcing awkward compromises. We propose Subpixel Placement of Tokens (SPoT), a novel tokenization strategy that positions tokens continuously within images, effectively sidestepping grid-based limitations. With our proposed oracle-guided search, we uncover substantial performance gains achievable with ideal subpixel token positioning, drastically reducing the number of tokens necessary for accurate predictions during inference. SPoT provides a new direction for flexible, efficient, and interpretable ViT architectures, redefining sparsity as a strategic advantage rather than an imposed limitation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Extracting full-field subpixel structural displacements from videos via deep learning
This paper develops a deep learning framework based on convolutional neural networks (CNNs) that enable real-time extraction of full-field subpixel structural displacements from videos. In particular, two new CNN archite…
Deep LearningSPOT: Sparsification with Attention Dynamics via Token Relevance in Vision Transformers
While Vision Transformers (ViT) have demonstrated remarkable performance across diverse tasks, their computational demands are substantial, scaling quadratically with the number of processed tokens. Compact attention rep…
Computational EfficiencyCoMatch: Dynamic Covisibility-Aware Transformer for Bilateral Subpixel-Level Semi-Dense Image Matching
This prospective study proposes CoMatch, a novel semi-dense image matcher with dynamic covisibility awareness and bilateral subpixel accuracy. Firstly, observing that modeling context interaction over the entire coarse f…
Computational EfficiencySpotEdit: Selective Region Editing in Diffusion Transformers
Diffusion Transformer models have significantly advanced image editing by encoding conditional images and integrating them into transformer layers. However, most edits involve modifying only small regions, while current …
Image EditingImage Registration Based Flicker Solving in Video Face Replacement and Analysis Based Sub-pixel Image Registration
In this paper, a framework of video face replacement is proposed and it deals with the flicker of swapped face in video sequence. This framework contains two main innovations: 1) the technique of image registration is ex…
Image Registration