TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding

2026-05-30 · Zhuoyu Wang, Junnan Huang, Xinyu Chen arxiv

Using a diffusion model for parallel drafting is a promising approach for speculative decoding. By predicting tokens at multiple future positions in a single forward pass, diffusion drafters substantially reduce drafting latency. However, this shifts the bottleneck to verification: verifying a single sequence limits acceptance length, while verifying large draft trees incurs excessive target-model latency. We identify a key mismatch in existing draft-tree methods: existing diffusion-tree methods rank nodes by the marginal probability, ignoring that verification is prefix-conditioned. As a result, they may verify unreachable descendants of rejected prefixes, increasing latency with limited acceptance gains. To address this, we propose TAPS, a target-aware prefix selection method that turns diffusion marginals into path-conditioned acceptance estimates. TAPS then selects a compact prefix-closed subtree under a fixed verification budget, improving the acceptance-cost tradeoff rather than simply expanding the draft tree. Experiments across diverse datasets and model families demonstrate that TAPS achieves up to 7.9x lossless end-to-end speedup over vanilla autoregressive decoding, outperforming state-of-the-art DFlash and DDTree by 1.36x and 1.74x respectively. Our work is available at https://anonymous.4open.science/r/TAPS-EMNLP2026-53DD

📄 PDF Abstract BibTeX arXiv:2606.00487

Code (0)

등록된 구현이 없습니다.

Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference

2026-05-07 · Saksham Rathi, Preeti, Mythili Vutukuru arxiv

Auto-regressive token generation in large language models is memory-bound because it requires "attending to" key and value tensors (KV cache) of all previous tokens. Prior work aims to improve the efficiency of this deco…

Reinforcement Learning

MetaPS: Adaptive Programmatic Strategy Selection for Market Agents

2026-06-21 · Jiaxiang Chen, Aotian Luo, Zhouyi Zheng, Weiyi Huang 외 arxiv

No single market strategy always wins: momentum, mean reversion, risk control,and event-driven rules can each succeed or fail as market conditions change.Rather than asking large language models to directly generate mark…

Difficulty-Aware Semantic-ID Optimization for Generative Recommendation

2026-08-20 · Xin Yu, Stephen Li, Sina Aghaei, Zifan Zhu 외 arxiv

Semantic-ID-based generative recommendation casts retrieval and ranking as autoregressive generation over hierarchical item identifiers. A common recipe is SFT followed by GRPO, yet vanilla GRPO is poorly matched to this…

LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training

2026-05-29 · Argyrios Gerogiannis, Yekaterina Yegorova, Mark Hasegawa-Johnson, Venugopal V. Veeravalli arxiv

State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to every token in a response. This ignores use…

Question Answering

Stochastic Polyak Stepsize with a Moving Target

2021-06-22 · Robert M. Gower, Aaron Defazio, Michael Rabbat

We propose a new stochastic gradient method called MOTAPS (Moving Targetted Polyak Stepsize) that uses recorded past loss values to compute adaptive stepsizes. MOTAPS can be seen as a variant of the Stochastic Polyak (SP…

image-classification Image Classification Translation