Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data. We present a controlled study that varies one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting. We find that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful. Transfer depends strongly on the origin relationship between teacher and student: same-origin pairs bring the student close to the teacher across languages, reasoning horizons, and even other domains, whereas cross-origin pairs mostly fit the trained distribution. This broad reach is a double-edged sword: since routing prompts to domain experts cannot confine each teacher's influence, combining them yields a mixture-dependent seesaw among their capabilities. These results clarify when OPD generalizes and offer a useful perspective for diagnosing multi-teacher OPD.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Automated Coin Recognition System using ANN
Coins are integral part of our day to day life. We use coins everywhere like grocery store, banks, buses, trains etc. So it becomes a basic need that coins can be sorted and counted automatically. For this it is necessar…
Two Facets of the Same Optimization Coin: Model Degradation and Representation Collapse in Graph Foundation Models
Inspired by the success of LLMs, GFMs are designed to learn the optimal embedding functions from multi-domain text-attributed graphs for the downstream cross-task generalization capability. Among the diverse architecture…
Domain GeneralizationGetting By Goal Misgeneralization With a Little Help From a Mentor
While reinforcement learning (RL) agents often perform well during training, they can struggle with distribution shift in real-world deployments. One particularly severe risk of distribution shift is goal misgeneralizati…
Reinforcement Learning (RL)Topic Modeling Based on Two-Step Flow Theory: Application to Tweets about Bitcoin
Digital cryptocurrencies such as Bitcoin have exploded in recent years in both popularity and value. By their novelty, cryptocurrencies tend to be both volatile and highly speculative. The capricious nature of these coin…
Heuristics for Detecting CoinJoin Transactions on the Bitcoin Blockchain
This research delves into the intricacies of Bitcoin, a decentralized peer-to-peer network, and its associated blockchain, which records all transactions since its inception. While this ensures integrity and transparency…