SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding
Diffusion large language models (DLLMs) enable non-autoregressive generation by iteratively denoising corrupted token sequences with bidirectional context. Despite their ability to update multiple positions in parallel, inference remains costly due to the many denoising steps required for high-quality generation. We propose SAID, a Scaffold-Aware Iterative Decoding framework that accelerates DLLMs by reallocating computation across tokens. SAID first spends denoising computation on scaffold tokens to establish the coarse semantic structure, and then completes predictable detail tokens with fewer steps. We further adapt SAID to block-wise diffusion decoding and introduce Confidence-Hierarchical Layered Generation (CHLG), which assigns additional steps only to low-confidence tokens. Experiments on LLaDA-8B and LLaDA 1.5 across math, coding, and knowledge benchmarks show that SAID significantly accelerates DLLM inference with a maximum speedup of 9.1x while maintaining competitive performance. Our code is publicly available: https://github.com/TH-AI-Lab-PKU/SAID.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Scaffold Diffusion: Sparse Multi-Category Voxel Structure Generation with Discrete Diffusion
Generating realistic sparse multi-category 3D voxel structures is difficult due to the cubic memory scaling of voxel structures and moreover the significant class imbalance caused by sparsity. We introduce Scaffold Diffu…
Autoregressive fragment-based diffusion for pocket-aware ligand design
In this work, we introduce AutoFragDiff, a fragment-based autoregressive diffusion model for generating 3D molecular structures conditioned on target protein structures. We employ geometric vector perceptrons to predict …
Scaffold-Aware Generative Augmentation and Reranking for Enhanced Virtual Screening
Ligand-based virtual screening (VS) is an essential step in drug discovery that evaluates large chemical libraries to identify compounds that potentially bind to a therapeutic target. However, VS faces three major challe…
Drug DiscoveryDiffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem
Construction of a scaffold structure that supports a desired motif, conferring protein function, shows promise for the design of vaccines and enzymes. But a general solution to this motif-scaffolding problem remains open…
Graph Neural NetworkDiffHopp: A Graph Diffusion Model for Novel Drug Design via Scaffold Hopping
Scaffold hopping is a drug discovery strategy to generate new chemical entities by modifying the core structure, the \emph{scaffold}, of a known active compound. This approach preserves the essential molecular features o…
Drug DesignDrug Discovery