Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments

2023-10-16 · Guanlin Meng

This paper aims to explore the potential of combining Deep Reinforcement Learning (DRL) with Knowledge Distillation (KD) by distilling various DRL algorithms and studying their distillation effects. By doing so, the computational burden of deep models could be reduced while maintaining the performance. The primary objective is to provide a benchmark for evaluating the performance of different DRL algorithms that have been refined using KD techniques. By distilling these algorithms, the goal is to develop efficient and fast DRL models. This research is expected to provide valuable insights that can facilitate further advancements in this promising direction. By exploring the combination of DRL and KD, this work aims to promote the development of models that require fewer GPU resources, learn more quickly, and make faster decisions in complex environments. The results of this research have the capacity to significantly advance the field of DRL and pave the way for the future deployment of resource-efficient, decision-making intelligent systems.

📄 PDF Abstract BibTeX arXiv:2310.10170

Code (1)

paopaolin/papercode/tree/main/MENGGUANLIN_papercode/combine%20V1 공식 구현

Tasks

Decision MakingDeep Reinforcement LearningGPUKnowledge Distillation reinforcement-learning

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Knowledge Distillation for Large Language Models

2026-03-14 · Alejandro Paredes La Torre, Barbara Flores, Diego Rodriguez arxiv

We propose a resource-efficient framework for compressing large language models through knowledge distillation, combined with guided chain-of-thought reinforcement learning. Using Qwen 3B as the teacher and Qwen 0.5B as …

Knowledge Distillation Reinforcement Learning

KD-MARL: Resource-Aware Knowledge Distillation in Multi-Agent Reinforcement Learning

2026-04-08 · Monirul Islam Pavel, Siyi Hu, Muhammad Anwar Masum, Mahardhika Pratama 외 arxiv

Real world deployment of multi agent reinforcement learning MARL systems is fundamentally constrained by limited compute memory and inference time. While expert policies achieve high performance they rely on costly decis…

Multi-agent Reinforcement Learning Knowledge Distillation

Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems

2026-03-27 · Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank 외 arxiv

Transformer-based reinforcement learning has emerged as a strong candidate for sequential control in residential energy management. In particular, the Decision Transformer can learn effective battery dispatch policies fr…

Knowledge Distillation Reinforcement Learning

Letz Translate: Low-Resource Machine Translation for Luxembourgish

2023-03-02 · Yewei Song, Saad Ezzini, Jacques Klein, Tegawende Bissyande 외

Natural language processing of Low-Resource Languages (LRL) is often challenged by the lack of data. Therefore, achieving accurate machine translation (MT) in a low-resource environment is a real problem that requires pr…

Knowledge Distillation Machine Translation Translation

Remote Sensing Image Classification with Decoupled Knowledge Distillation

2025-05-25 · Yaping He, JianFeng Cai, Qicong Hu, Peiqing Wang

To address the challenges posed by the large number of parameters in existing remote sensing image classification models, which hinder deployment on resource-constrained devices, this paper proposes a lightweight classif…

Classification image-classification Image Classification Knowledge Distillation+1