홈 › Papers

Distilling Policy Distillation

2019-02-06 · Wojciech Marian Czarnecki, Razvan Pascanu, Simon Osindero, Siddhant M. Jayakumar, Grzegorz Swirszcz, Max Jaderberg

The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success, for example, by enhancing the optimisation of agents, leading to stronger performance faster, on harder domains [26, 32, 5, 8]. Despite the widespread use and conceptual simplicity of distillation, many different formulations are used in practice, and the subtle variations between them can often drastically change the performance and the resulting objective that is being optimised. In this work, we rigorously explore the entire landscape of policy distillation, comparing the motivations and strengths of each variant through theoretical and empirical analysis. Our results point to three distillation techniques, that are preferred depending on specifics of the task. Specifically a newly proposed expected entropy regularised distillation allows for quicker learning in a wide range of situations, while still guaranteeing convergence.

📄 PDF Abstract BibTeX arXiv:1902.02186

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningReinforcement Learning

Distilling Reinforcement Learning into Single-Batch Datasets

2025-08-12 · Connor Wilhelm, Dan Ventura arxiv

Dataset distillation compresses a large dataset into a small synthetic dataset such that learning on the synthetic dataset approximates learning on the original. Training on the distilled dataset can be performed in as l…

Reinforcement Learning Atari Games

Proximal Policy Distillation

2024-07-21 · Giacomo Spigler

We introduce Proximal Policy Distillation (PPD), a novel policy distillation method that integrates student-driven distillation and Proximal Policy Optimization (PPO) to increase sample efficiency and to leverage the add…

continuous-control Continuous Control MuJoCo

Real-time Policy Distillation in Deep Reinforcement Learning

2019-12-29 · Yuxiang Sun, Pooyan Fazli

Policy distillation in deep reinforcement learning provides an effective way to transfer control policies from a larger network to a smaller untrained network without a significant degradation in performance. However, po…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

On-Policy Adversarial Flow Distillation for Autoregressive Video Generation

2026-05-25 · Yang Luo, Shengju Qian, Xiaohang Tang, Zirui Zhu 외 arxiv

Autoregressive video generators are attractive for streaming, long-horizon, and interactive applications, but distilling strong black-box teachers into causal students remains difficult. The student must learn under its …

Reinforcement Learning Video Generation

OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning

2026-05-12 · Yuxiao Yang, Xiaoyun Wang, Weitong Zhang arxiv

We study on-policy self-distillation (OPSD), where a language model improves its reasoning ability by distilling privileged teacher distributions along its own on-policy trajectories. Despite its promise, OPSD can suffer…

Mathematical Reasoning