ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

2026-06-29 · Zilong Liu, Xuewen Zhang, Jinrui Xing, Juyi Qiao, Huiyong Wang, Junming Jiao arxiv

Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limiting both generation quality and generalization. To address this, we analyze the complementary roles of forward and reverse KL divergence (FKL/RKL) in distribution alignment from theoretical and empirical perspectives. We then propose a reinforcement-learning-based adaptive KL-weighted distillation framework, in which a policy network dynamically assigns weights to FKL and RKL based on teacher-student distributional characteristics, guided by immediate reward signals to achieve dual alignment on principal and long-tail modes. Extensive experiments demonstrate consistent improvements across Rouge-L and BertScore metrics, surpassing greedy heuristics by 0.4-0.6 points and outperforming other baseline methods on diverse benchmarks.

📄 PDF Abstract BibTeX arXiv:2606.29869

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Knowledge Distillation Text Generation

FMBench: Adaptive Large Language Model Output Formatting

2026-02-06 · Yaoting Wang, Yun Zhou, Henghui Ding arxiv

Producing outputs that satisfy both semantic intent and format constraints is essential for deploying large language models in user-facing and system-integrated workflows. In this work, we focus on Markdown formatting, w…

Reinforcement Learning

DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers

2022-04-27 · CVPR 2022 1 · Xianing Chen, Qiong Cao, Yujie Zhong, Jing Zhang 외

Transformers are successfully applied to computer vision due to their powerful modeling capacity with self-attention. However, the excellent performance of transformers heavily depends on enormous training images. Thus, …

Knowledge Distillation

GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation

2026-05-12 · Sijia Li, Yuchen Huang, Zifan Liu, Yanping Li 외 arxiv

Reinforcement learning has become a widely used post-training approach for LLM agents, where training commonly relies on outcome-level rewards that provide only coarse supervision. While finer-grained credit assignment i…

Reinforcement Learning Mathematical Reasoning

Distilling Autoregressive Models to Obtain High-Performance Non-Autoregressive Solvers for Vehicle Routing Problems with Faster Inference Speed

2023-12-19 · Yubin Xiao, Di Wang, Boyang Li, Mingzhao Wang 외

Neural construction models have shown promising performance for Vehicle Routing Problems (VRPs) by adopting either the Autoregressive (AR) or Non-Autoregressive (NAR) learning approach. While AR models produce high-quali…

Knowledge Distillation

Variational Inference with Tail-adaptive f-Divergence

2018-10-29 · NeurIPS 2018 12 · Dilin Wang, Hao liu, Qiang Liu

Variational inference with {\alpha}-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using {\alpha}-divergences (with positive …

Deep Reinforcement Learning Reinforcement Learning Variational Inference