Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search

2020-12-24 · Yonggan Fu, Zhongzhi Yu, Yongan Zhang, Yingyan Celine Lin

AlphaGo's astonishing performance has ignited an explosive interest in developing deep reinforcement learning (DRL) for numerous real-world applications, such as intelligent robotics. However, the often prohibitive complexity of DRL stands at the odds with the required real-time control and constrained resources in many DRL applications, limiting the great potential of DRL powered intelligent devices. While substantial efforts have been devoted to compressing other deep learning models, existing works barely touch the surface of compressing DRL. In this work, we first identify that there exists an optimal model size of DRL that can maximize both the test scores and efficiency, motivating the need for task-specific DRL agents. We therefore propose an Auto-Agent-Distiller (A2D) framework, which to our best knowledge is the first neural architecture search (NAS) applied to DRL to automatically search for the optimal DRL agents for various tasks that optimize both the test scores and efficiency. Specifically, we demonstrate that vanilla NAS can easily fail in searching for the optimal agents, due to its resulting high variance in DRL training stability, and then develop a novel distillation mechanism to distill the knowledge from both the teacher agent's actor and critic to stabilize the searching process and improve the searched agents' optimality. Extensive experiments and ablation studies consistently validate our findings and the advantages and general applicability of our A2D, outperforming manually designed DRL in both the test scores and efficiency. All the codes will be released upon acceptance.

📄 PDF Abstract BibTeX arXiv:2012.13091

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningNeural Architecture Search reinforcement-learning Reinforcement Learning (RL)

CodeDistiller: Automatically Generating Code Libraries for Scientific Coding Agents

2025-11-30 · Peter Jansen, Samiah Hassan, Pragnya Narasimha arxiv

Automated Scientific Discovery (ASD) systems can help automatically generate and run code-based experiments, but their capabilities are limited by the code they can reliably generate from parametric knowledge alone. As a…

KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs

2023-09-21 · NeurIPS 2023 11

Knowledge distillation (KD) has emerged as an effective technique for compressing models that can enhance the lightweight model. Conventional KD methods propose various designs to allow student model to imitate the teac…

Decentralized scheduling through an adaptive, trading-based multi-agent system

2022-07-05 · Michael Kölle, Lennart Rietdorf, Kyrill Schmid

In multi-agent reinforcement learning systems, the actions of one agent can have a negative impact on the rewards of other agents. One way to combat this problem is to let agents trade their rewards amongst each other. M…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning

2019-05-31 · Matthew A. Wright, Roberto Horowitz

Many potential applications of reinforcement learning in the real world involve interacting with other agents whose numbers vary over time. We propose new neural policy architectures for these multi-agent problems. In co…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Self Training Autonomous Driving Agent

2019-04-26 · Shashank Kotyan, Danilo Vasconcellos Vargas, Venkanna U

Intrinsically, driving is a Markov Decision Process which suits well the reinforcement learning paradigm. In this paper, we propose a novel agent which learns to drive a vehicle without any human assistance. We use the c…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)