dLLM: Simple Diffusion Language Modeling

2026-02-26 · Zhanhui Zhou, Lingjie Chen, Hanghang Tong, Dawn Song arxiv

Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.

📄 PDF Abstract BibTeX arXiv:2602.22661

Code (0)

등록된 구현이 없습니다.

SimSD: Simple Speculative Decoding in Diffusion Language Models

2026-06-01 · Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo 외 arxiv

Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding. However, their masked language model…

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

2026-01-25 · Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo, Yong Luo 외 arxiv

Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressiv…

Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model

2026-01-22 · Chenghao Fan, Wen Heng, Bo Li, Sichen Liu 외 arxiv

Diffusion-based language models (DLLMs) offer non-sequential, block-wise generation and richer data reuse compared to autoregressive (AR) models, but existing code DLLMs still lag behind strong AR baselines under compara…

Continual Pretraining Data Augmentation

Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing

2025-08-08 · Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin 외 arxiv

Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs for text generation, with the potential to decode multiple tokens in a single iteration. However, none of the ex…

Text Generation

Fast-dLLM v2: Efficient Block-Diffusion LLM

2025-09-30 · Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao 외 arxiv

Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency. In this work, we pr…