BackPlay: Head-Only Look-Back Self-Correction for Diffusion Language Models

2026-01-10 · Liming Liu, Binxuan Huang, Zixuan Zhang, Xin Liu, Bing Yin, Tuo Zhao arxiv

Diffusion Language Models (DLMs) decode multiple tokens in parallel, but aggressive multi-token decoding amplifies cross-token dependency errors and can sharply degrade generation quality. We propose BackPlay, a frozen-backbone self-correction framework that trains only a lightweight correction head on a finetuned DLM without updating any backbone or adapter parameters. Because the head is trained on errors produced by the same frozen generator used at inference time, its training distribution aligns with the error patterns of the deployed model. We further introduce Look-back Correction, a training mechanism that injects predictions from earlier, more corrupted denoising states into later, richer contexts, enabling the head to leverage later context to detect mistakes made in earlier generation steps. During inference, BackPlay periodically revisits previously generated tokens through selective remasking and regeneration to limit error accumulation. Across mathematical reasoning and code generation benchmarks, BackPlay improves the speed--quality trade-off of the underlying DLM under multi-token decoding.

📄 PDF Abstract BibTeX arXiv:2601.06428

Code (0)

등록된 구현이 없습니다.

Tasks

Mathematical Reasoning Code Generation

Backplay: "Man muss immer umkehren"

2018-07-18 · Cinjon Resnick, Roberta Raileanu, Sanyam Kapoor, Alexander Peysakhovich 외

Model-free reinforcement learning (RL) requires a large number of trials to learn a good policy, especially in environments with sparse rewards. We explore a method to improve the sample efficiency when we have access to…

Reinforcement Learning Reinforcement Learning (RL)

Backplay: 'Man muss immer umkehren'

2019-05-01 · ICLR 2019 5 · Cinjon Resnick, Roberta Raileanu, Sanyam Kapoor, Alexander Peysakhovich 외

Reinforcement Learning Reinforcement Learning (RL)

PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning

2020-04-24 · Guillaume Matheron, Nicolas Perrin, Olivier Sigaud

The exploration-exploitation trade-off is at the heart of reinforcement learning (RL). However, most continuous control benchmarks used in recent RL research only require local exploration. This led to the development of…

continuous-control Continuous Control Efficient Exploration Motion Planning+3

Gradient Extrapolation-Based Policy Optimization

2026-05-07 · Ismam Nur Swapnil, Aranya Saha, Tanvir Ahmed Khan, Mohammad Ariful Haque 외 arxiv

Reinforcement learning is widely used to improve the reasoning ability of large language models, especially when answers can be automatically checked. Standard GRPO-style training updates the model using only the current…

Reinforcement Learning

Reconcile Once, Write Anytime: A Trust-Tiered Librarian and a Multi-Agent Writer for Drift-Free, Point-in-Time Research

2026-08-13 · Xing Zhang, Yanwei Cui, Guanghui Wang, Peiyang He arxiv

Long-form research reports generated by large language models drift, contradict themselves, and lose provenance: the same metric appears with different values, and rumor is quoted as confidently as an audited filing. We …