MixDiT: Accelerating Image Diffusion Transformer Inference with Mixed-Precision MX Quantization
Diffusion Transformer (DiT) has driven significant progress in image generation tasks. However, DiT inferencing is notoriously compute-intensive and incurs long latency even on datacenter-scale GPUs, primarily due to its iterative nature and heavy reliance on GEMM operations inherent to its encoder-based structure. To address the challenge, prior work has explored quantization, but achieving low-precision quantization for DiT inferencing with both high accuracy and substantial speedup remains an open problem. To this end, this paper proposes MixDiT, an algorithm-hardware co-designed acceleration solution that exploits mixed Microscaling (MX) formats to quantize DiT activation values. MixDiT quantizes the DiT activation tensors by selectively applying higher precision to magnitude-based outliers, which produce mixed-precision GEMM operations. To achieve tangible speedup from the mixed-precision arithmetic, we design a MixDiT accelerator that enables precision-flexible multiplications and efficient MX precision conversions. Our experimental results show that MixDiT delivers a speedup of 2.10-5.32 times over RTX 3090, with no loss in FID.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NAMI: Efficient Image Generation via Progressive Rectified Flow Transformers
Flow-based transformer models for image generation have achieved state-of-the-art performance with larger model parameters, but their inference deployment cost remains high. To enhance inference performance while maintai…
Image GenerationDuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach
Diffusion models have achieved unprecedented performance in image generation, yet they suffer from slow inference due to their iterative sampling process. To address this, early-exiting has recently been proposed, where …
DenoisingImage GenerationEvolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model
Diffusion-based image generation models excel at producing high-quality synthetic content, but suffer from slow and computationally expensive inference. Prior work has attempted to mitigate this by caching and reusing fe…
Image GenerationSPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference
Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost, making inference prohibitive at video-token scales. The challenge is input-adaptive sparsity: selecting critical Q/K/V tok…
Video GenerationDiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
Diffusion models demonstrate outstanding performance in image generation, but their multi-step inference mechanism requires immense computational cost. Previous works accelerate inference by leveraging layer or token cac…
Image Generation