Dream 7B: Diffusion Large Language Models

2025-08-21 · Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong arxiv

We introduce Dream 7B, the most powerful open diffusion large language model to date. Unlike autoregressive (AR) models that generate tokens sequentially, Dream 7B employs discrete diffusion modeling to refine sequences in parallel through iterative denoising. Our model consistently outperforms existing diffusion language models on general, mathematical, and coding tasks. Dream 7B demonstrates superior planning abilities and inference flexibility, including arbitrary-order generation, infilling capabilities, and tunable quality-speed trade-offs. These results are achieved through simple yet effective training techniques, including AR-based LLM initialization and context-adaptive token-level noise rescheduling. We release both Dream-Base and Dream-Instruct to facilitate further research in diffusion-based language modeling.

📄 PDF Abstract BibTeX arXiv:2508.15487

Code (0)

등록된 구현이 없습니다.

Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone

2025-12-27 · Jiacheng Ye, Shansan Gong, Jiahui Gao, Junming Fan 외 arxiv

While autoregressive Large Vision-Language Models (VLMs) have achieved remarkable success, their sequential generation often limits their efficacy in complex visual planning and dynamic robotic control. In this work, we …

DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models

2026-06-17 · Zirui Wu, Lin Zheng, Jiacheng Ye, Shansan Gong 외 arxiv

Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains unresolved. To this end, we develop Dre…

DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas

2026-02-01 · Zirui Wu, Lin Zheng, Zhihui Xie, Jiacheng Ye 외 arxiv

Diffusion Language Models (DLMs) present a compelling alternative to autoregressive models, offering flexible, any-order infilling without specialized prompting design. However, their practical utility is blocked by a cr…

Dream-Coder 7B: An Open Diffusion Language Model for Code

2025-09-01 · Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao 외 arxiv

We present Dream-Coder 7B, an open-source discrete diffusion language model for code generation that exhibits emergent any-order generation capabilities. Unlike traditional autoregressive (AR) models that decode strictly…

Reinforcement Learning Code Generation

DreamReader: An Interpretability Toolkit for Text-to-Image Models

2026-03-02 · Nirmalendu Prakash, Narmeen Oozeer, Michael Lan, Luka Samkharadze 외 arxiv

Despite the rapid adoption of text-to-image (T2I) diffusion models, causal and representation-level analysis remains fragmented and largely limited to isolated probing techniques. To address this gap, we introduce DreamR…