Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation

2024-11-25 · Muhammad Burhan Hafez, Kerim Erekmen

Central to the development of universal learning systems is the ability to solve multiple tasks without retraining from scratch when new data arrives. This is crucial because each task requires significant training time. Addressing the problem of continual learning necessitates various methods due to the complexity of the problem space. This problem space includes: (1) addressing catastrophic forgetting to retain previously learned tasks, (2) demonstrating positive forward transfer for faster learning, (3) ensuring scalability across numerous tasks, and (4) facilitating learning without requiring task labels, even in the absence of clear task boundaries. In this paper, the Task-Agnostic Policy Distillation (TAPD) framework is introduced. This framework alleviates problems (1)-(4) by incorporating a task-agnostic phase, where an agent explores its environment without any external goal and maximizes only its intrinsic motivation. The knowledge gained during this phase is later distilled for further exploration. Therefore, the agent acts in a self-supervised manner by systematically seeking novel states. By utilizing task-agnostic distilled knowledge, the agent can solve downstream tasks more efficiently, leading to improved sample efficiency. Our code is available at the repository: https://github.com/wabbajack1/TAPD.

📄 PDF Abstract BibTeX arXiv:2411.16532

Code (1)

wabbajack1/tapd 공식 구현 pytorch

Tasks

Continual LearningDeep Reinforcement Learningreinforcement-learning Reinforcement Learning

Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer

2019-06-11 · René Traoré, Hugo Caselles-Dupré, Timothée Lesort, Te Sun 외

We focus on the problem of teaching a robot to solve tasks presented sequentially, i.e., in a continual learning scenario. The robot should be able to solve all tasks it has encountered, without forgetting past tasks. We…

Continual Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

DisCoRL: Continual Reinforcement Learning via Policy Distillation

2019-07-11 · René Traoré, Hugo Caselles-Dupré, Timothée Lesort, Te Sun 외

In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an exter…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

Continual Policy Distillation from Distributed Reinforcement Learning Teachers

2026-01-30 · Yuxuan Li, Qijun He, Mingqi Yuan, Wen-Tse Chen 외 arxiv

Continual Reinforcement Learning (CRL) aims to develop lifelong learning agents to continuously acquire knowledge across diverse tasks while mitigating catastrophic forgetting. This requires efficiently managing the stab…

Reinforcement Learning Multi-Task Learning

Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation

2024-04-05 · Lanpei Li, Enrico Donato, Vincenzo Lomonaco, Egidio Falotico

Dexterous manipulation, often facilitated by multi-fingered robotic hands, holds solid impact for real-world applications. Soft robotic hands, due to their compliant nature, offer flexibility and adaptability during obje…

Reinforcement Learning (RL)

Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training

2026-07-02 · Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang 외 arxiv

Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation em…

Reinforcement Learning Continual Learning