Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models

2026-07-05 · Lin Yao arxiv

Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-$n$ layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

📄 PDF Abstract BibTeX arXiv:2607.04469

Code (0)

등록된 구현이 없습니다.

TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

2026-07-03 · Chengcheng Wang, Tingzhang Luo, Wenhao Li, Jianyuan Guo 외 arxiv

Diffusion language models (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this t…

The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models

2026-05-23 · Bohang Sun, Max Zhu, Francesco Caso, Jindong Gu 외 arxiv

Diffusion large language models promise faster generation by refining many token positions in parallel, but this parallelism introduces a hidden control problem: which proposed tokens should be transferred into the parti…

Mathematical Reasoning Question Answering Code Generation

Deferred Commitment Decoding for Diffusion Language Models

2026-01-05 · Yingte Shu, Yuchuan Tian, Chao Xu, Yunhe Wang 외 arxiv

Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV-cache compatibility, prior work commonl…

Code Generation Text Generation

Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs

2026-08-27 · Xingyou Fang, Jingxing Zhong, Xiaosong Yuan, Xiaofeng Zhang arxiv

Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence-based strategies prioritize locally easy tokens, but confidence doe…

When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models

2026-06-19 · Peihong Li, Yuanjie Shi, Yan Yan arxiv

Standard tokenwise diffusion LMs keep training corruption and inference commitment at token granularity throughout denoising. At high noise, this leaves scattered local fragments rather than coherent evidence, making it …