ToDo: Token Downsampling for Efficient Generation of High-Resolution Images
Attention mechanism has been crucial for image diffusion models, however, their quadratic computational complexity limits the sizes of images we can process within reasonable time and memory constraints. This paper investigates the importance of dense attention in generative image models, which often contain redundant features, making them suitable for sparser attention mechanisms. We propose a novel training-free method ToDo that relies on token downsampling of key and value tokens to accelerate Stable Diffusion inference by up to 2x for common sizes and up to 4.5x or more for high resolutions like 2048x2048. We demonstrate that our approach outperforms previous methods in balancing efficient throughput and fidelity.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HieraTok: Multi-Scale Visual Tokenizer Improves Image Reconstruction and Generation
In this work, we present HieraTok, a novel multi-scale Vision Transformer (ViT)-based tokenizer that overcomes the inherent limitation of modeling single-scale representations. This is realized through two key designs: (…
Image ReconstructionImage GenerationTodoEvolve: Learning to Architect Agent Planning Systems
Planning has become a central capability for contemporary agent systems in navigating complex, long-horizon tasks, yet existing approaches predominantly rely on fixed, hand-crafted planning structures that lack the flexi…
Reinforcement LearningBenchmarking and Boosting Radiology Report Generation for 3D High-Resolution Medical Images
Automatic radiology report generation can significantly benefit the labor-intensive process of report writing by radiologists, especially for 3D radiographs like CT scans, which are crucial for broad clinical diagnostics…
BenchmarkingGPUNanoFLUX: Distillation-Driven Compression of Large Text-to-Image Generation Models for Mobile Devices
While large-scale text-to-image diffusion models continue to improve in visual quality, their increasing scale has widened the gap between state-of-the-art models and on-device solutions. To address this gap, we introduc…
Text-to-Image GenerationModel CompressionIncreasing the Efficiency of DETR for Maritime High-Resolution Images
Maritime object detection is critical for the safe navigation of unmanned surface vessels (USVs), requiring accurate recognition of obstacles from small buoys to large vessels. Real-time detection is challenging due to l…
Computational EfficiencyObject Detection