FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving

2026-07-13 · Yaqi Qiao, Ping He, Songrun Xie, Ayush Barik, Chensong Zhang, Zhengzhong Tu, Fan Lai arxiv

Diffusion models have become the central backbone for modern image, video, and audio generation, but their efficient service remains a challenge. Unlike autoregressive decoding, diffusion inference repeatedly updates high-dimensional spatial or temporal latents over many denoising steps. This all-region execution pattern makes generation latency high and limits serving throughput. Existing multi-GPU parallelization methods can reduce per-step computation, but often introduce substantial activation exchange overhead, causing communication to offset or even outweigh the benefits of parallel execution. This paper presents FlashDiff, a diffusion serving system that improves inference efficiency through adaptive regional execution and scheduling. FlashDiff is based on the observation that diffusion refinement is not uniform across latent regions or denoising steps: different regions often stabilize at different rates, while neighboring steps exhibit strong temporal correlation. FlashDiff leverages these properties to selectively execute only regions that require further refinement and to reallocate the resulting compute slack across concurrent serving requests. FlashDiff consists of three mechanisms. First, it decomposes the latent representation into coherent execution regions using early-stage attention signals, preserving semantic structure while exposing fine-grained parallelism. Second, it uses a lightweight runtime controller to estimate region activity and bypass low-impact updates when further refinement is unlikely to affect output quality. Third, it applies an affinity-aware online scheduler that co-locates dependent regions, balances residual load across GPUs, and reuses reclaimed compute capacity to improve serving efficiency. Across real-world image, video, and audio workloads, FlashDiff reduces end-to-end serving latency by 30-97% and improves throughput by 1.2-2.2x.

📄 PDF Abstract BibTeX arXiv:2607.12121

Code (0)

등록된 구현이 없습니다.

Tasks

Audio Generation

SDiT: Semantic Region-Adaptive for Diffusion Transformers

2026-01-18 · Bowen Lin, Fanjiang Ye, Yihua Liu, Zhenghui Guo 외 arxiv

Diffusion Transformers (DiTs) achieve state-of-the-art performance in text-to-image synthesis but remain computationally expensive due to the iterative nature of denoising and the quadratic cost of global attention. In t…

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

2026-07-09 · Yuanjie Zhu, Liangwei Yang, Ke Xu, Weizhi Zhang 외 arxiv

Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall be…

GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving

2026-06-11 · Xinwei Qiang, Yifan Hu, Shixuan Sun, Jing Yang 외 arxiv

Diffusion Transformers (DiTs) have become the dominant architecture for image and video generation, creating growing demand for efficient DiT serving. Existing systems assign each request a fixed parallel configuration t…

Video Generation

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

2026-07-05 · Nitin Kedia, Saurabh Agarwal, Myungjin Lee, Aditya Akella arxiv

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style …

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency

2025-05-20 · Ruixiao Li, Fahao Chen, Peng Li

Speculative decoding accelerates Large Language Model (LLM) inference by employing a small speculative model (SSM) to generate multiple candidate tokens and verify them using the LLM in parallel. This technique has been …

Large Language Model Scheduling