홈 › Papers

FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation

2026-01-30 · Siyang He, Qiqi Wang, Xiaoran Liu, Hongnan Ma, Yiwei Shi, Yuerong Song, Ying Zhu, Tianyi Liang, Zengfeng Huang, Ziwei He, Xipeng Qiu arxiv

Despite the non-autoregressive potential of diffusion language models (dLLMs), existing decoding strategies demonstrate positional bias, failing to fully unlock the potential of arbitrary generation. In this work, we delve into the inherent spectral characteristics of dLLMs and present the first frequency-domain analysis showing that low-frequency components in hidden states primarily encode global structural information and long-range dependencies, while high-frequency components are responsible for characterizing local details. Based on this observation, we propose FourierSampler, which leverages a frequency-domain sliding window mechanism to dynamically guide the model to achieve a "structure-to-detail" generation. FourierSampler outperforms other inference enhancement strategies on LLADA and SDAR, achieving relative improvements of 20.4% on LLaDA1.5-8B and 16.0% on LLaDA-8B-Instruct. It notably surpasses similarly sized autoregressive models like Llama3.1-8B-Instruct.

📄 PDF Abstract BibTeX arXiv:2601.23182

Code (0)

등록된 구현이 없습니다.

Unlocking the Potential of Diffusion Language Models through Template Infilling

2025-10-13 · Junhoo Lee, Seungyeon Kim, Nojun Kwak arxiv

Diffusion Language Models (DLMs) have emerged as a promising alternative to Autoregressive Language Models, yet their inference strategies remain limited to prefix-based prompting inherited from the autoregressive paradi…

Mathematical Reasoning Code Generation

Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Code

2026-04-09 · Kishan Maharaj, Ashita Saxena, Srikanth Tamilselvam arxiv

Autoregressive (AR) language models have driven significant progress in automated software engineering, enabling powerful code generation and assistance systems. However, the next-token prediction paradigm introduces str…

Code Generation

Causal Autoregressive Diffusion Language Model

2026-01-29 · Junhao Ruan, Bei Li, Yongjing Yin, Pengcheng Huang 외 arxiv

In this work, we propose Causal Autoregressive Diffusion (CARD), a novel framework that unifies the training efficiency of ARMs with the high-throughput inference of diffusion models. CARD reformulates the diffusion proc…

GeoDiT: A Diffusion-based Vision-Language Model for Geospatial Understanding

2025-12-02 · Jiaqi Liu, Ronghao Fu, Haoran Liu, Lang Sun 외 arxiv

Autoregressive models are structurally misaligned with the inherently parallel nature of geospatial understanding, forcing a rigid sequential narrative onto scenes and fundamentally hindering the generation of structured…

Visual Grounding Object Detection Image Captioning

Revisiting Autoregressive Models for Generative Image Classification

2026-03-19 · Ilia Sudakov, Artem Babenko, Dmitry Baranchuk arxiv

Class-conditional generative models have emerged as accurate and robust classifiers, with diffusion models demonstrating clear advantages over other visual generative paradigms, including autoregressive (AR) models. In t…

Image Classification