홈 › Papers

DARE: Diffusion Language Model Activation Reuse for Efficient Inference

2026-05-01 · Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang, Chi-Chih Chang, Mohamed S. Abdelfattah, Diana Marculescu arxiv

Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to auto-regressive (AR) models, offering greater expressive capacity and potential for parallel generation and faster inference. However, open-source dLLMs remain immature, lagging behind AR models in both efficiency and quality. We identify an underexplored property of dLLMs: *token-wise redundancy* in bi-directional self-attention. Self-attention activations are highly correlated across tokens, and temporal changes in query representations can predict redundancy in corresponding key, value, and output activations. We introduce DARE, with two complementary mechanisms: DARE-KV, which reuses cached key-value (KV) activations, and DARE-O, which reuses output activations to reduce redundant computation while preserving quality. DARE achieves up to 1.20x per-layer latency reduction and reuses up to 87% of attention activations, with negligible degradation on reasoning and code-generation benchmarks. DARE-KV and DARE-O incur average performance drops of only 2.0% and 1.2%, respectively. Combined with techniques such as prefix caching and Fast-dLLM, DARE provides additive gains without retraining. These results establish token-wise reuse as an effective strategy for improving the efficiency of diffusion-based LLMs while preserving generation fidelity. Code: https://github.com/enyac-group/DARE

📄 PDF Abstract BibTeX arXiv:2605.08134

Code (0)

등록된 구현이 없습니다.

DARE: Diffusion Large Language Models Alignment and Reinforcement Executor

2026-04-05 · Jingyi Yang, Yuxian Jiang, Xuhao Hu, Shuang Cheng 외 arxiv

Diffusion large language models (dLLMs) are emerging as a compelling alternative to dominant autoregressive models, replacing strictly sequential token generation with iterative denoising and parallel generation dynamics…

parameter-efficient fine-tuning Reinforcement Learning

Sparse ActionGen: Accelerating Diffusion Policy with Real-time Pruning

2026-01-19 · Kangye Ji, Jianbo Zhou, Yuan Meng, Ye Li 외 arxiv

Diffusion Policy has dominated action generation due to its strong capabilities for modeling multi-modal action distributions, but its multi-step denoising processes make it impractical for real-time visuomotor control. …

General and Efficient Steering of Diffusion Models

2026-02-11 · Qingsong Wang, Mikhail Belkin, Yusu Wang arxiv

Steering diffusion models toward conditions unseen during training typically requires either retraining with conditional inputs or per-step gradient computations, both of which incur substantial computational overhead. W…

AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers

2026-02-13 · Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu arxiv

Diffusion Transformers (DiTs) achieve state-of-the-art performance in high-fidelity image and video generation but suffer from expensive inference due to their iterative denoising structure. While prior methods accelerat…

Video Generation

MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers

2026-06-14 · Maoliang Li, Haojing Chen, Jiayu Chen, Zihao Zheng 외 arxiv

Diffusion Transformers with Mixture-of-Experts (DiT-MoE) improve model capacity under sparse activation, but diffusion inference is still bottlenecked by redundant computation across timesteps. Existing caching methods m…