Alignment-Aware Decoding

2025-09-30 · Frédéric Berdoz, Luca A. Lanzendörfer, René Caky, Roger Wattenhofer arxiv

Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving alignment, typically through training-time or prompt-based interventions. In this paper, we introduce alignment-aware decoding (AAD), a method to enhance model alignment directly at inference. Theoretically, AAD can be interpreted as implicit reward optimization, yet it requires no specialized training beyond the standard DPO setup. Empirically, AAD consistently outperforms strong baselines across diverse alignment benchmarks and model scales. Moreover, in data-constrained settings, AAD can produce high-quality synthetic data to improve alignment under standard decoding, providing a practical solution when labeled data is limited.

📄 PDF Abstract BibTeX arXiv:2509.26169

Code (0)

등록된 구현이 없습니다.

DARC: Disagreement-Aware Alignment via Risk-Constrained Decoding

2026-03-09 · Mingxi Zou, Jiaxiang Chen, Junfan Li, Langzhang Liang 외 arxiv

Preference-based alignment methods (e.g., RLHF, DPO) typically optimize a single scalar objective, implicitly averaging over heterogeneous human preferences. In practice, systematic annotator and user-group disagreement …

Decision Making

Fast Whole-Brain, Geometry-Aware Functional Alignment for Cross-Subject Decoding

2026-07-12 · Pierre-Louis Barbarant, Florent Meyniel, Bertrand Thirion arxiv

Decoding brain activity is useful for characterizing brain processes and understanding the functional architecture underlying cognition. However, the inter-individual variability in brain response patterns limits the dev…

Computational Efficiency

SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining

2026-02-11 · Yifan Zhang, Zunhai Su, Shuhao Hu, Rui Yang 외 arxiv

While FP8 attention has shown substantial promise in innovations like FlashAttention-3, its integration into the decoding phase of the DeepSeek Multi-head Latent Attention (MLA) architecture presents notable challenges. …

The Alignment Problem in Constrained Code Generation

2026-06-19 · Matteo Biagiola, Jahrim Gabriele Cesario, Luca Di Grazia, George Zakhour 외 arxiv

Large Language Models (LLMs) have demonstrated strong capabilities in code generation, but their outputs frequently contain syntax or type errors that result in compilation failures. Constrained decoding has been propose…

Code Generation

Steering Multimodal Large Language Models Decoding for Context-Aware Safety

2025-09-23 · Zheyuan Liu, Zhangchen Xu, Guangyao Dou, Xiangchi Yuan 외 arxiv

Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitiv…