Real-time Policy Distillation in Deep Reinforcement Learning

2019-12-29 · Yuxiang Sun, Pooyan Fazli

Policy distillation in deep reinforcement learning provides an effective way to transfer control policies from a larger network to a smaller untrained network without a significant degradation in performance. However, policy distillation is underexplored in deep reinforcement learning, and existing approaches are computationally inefficient, resulting in a long distillation time. In addition, the effectiveness of the distillation process is still limited to the model capacity. We propose a new distillation mechanism, called real-time policy distillation, in which training the teacher model and distilling the policy to the student model occur simultaneously. Accordingly, the teacher's latest policy is transferred to the student model in real time. This reduces the distillation time to half the original time or even less and also makes it possible for extremely small student models to learn skills at the expert level. We evaluated the proposed algorithm in the Atari 2600 domain. The results show that our approach can achieve full distillation in most games, even with compression ratios up to 1.7%.

📄 PDF Abstract BibTeX arXiv:1912.12630

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

DisCoRL: Continual Reinforcement Learning via Policy Distillation

2019-07-11 · René Traoré, Hugo Caselles-Dupré, Timothée Lesort, Te Sun 외

In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an exter…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer

2019-06-11 · René Traoré, Hugo Caselles-Dupré, Timothée Lesort, Te Sun 외

We focus on the problem of teaching a robot to solve tasks presented sequentially, i.e., in a continual learning scenario. The robot should be able to solve all tasks it has encountered, without forgetting past tasks. We…

Continual Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Low Precision Policy Distillation with Application to Low-Power, Real-time Sensation-Cognition-Action Loop with Neuromorphic Computing

2018-09-25 · Jeffrey L. McKinstry, Davis R. Barch, Deepika Bablani, Michael V. Debole 외

Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demon…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization

2022-07-29 · Yuki Kadokawa, Lingwei Zhu, Yoshihisa Tsurumine, Takamitsu Matsubara

Deep reinforcement learning with domain randomization learns a control policy in various simulations with randomized physical and sensor model parameters to become transferable to the real world in a zero-shot setting. H…

Deep Reinforcement Learning MuJoCo reinforcement-learning Reinforcement Learning (RL)+1

Practical Policy Distillation for Reinforcement Learning in Radio Access Networks

2025-11-09 · Sara Khosravi, Burak Demirel, Linghui Zhou, Javier Rasines 외 arxiv

Adopting artificial intelligence (AI) in radio access networks (RANs) presents several challenges, including limited availability of link-level measurements (e.g., CQI reports), stringent real-time processing constraints…

Reinforcement Learning