Building a Subspace of Policies for Scalable Continual Learning
The ability to continuously acquire new knowledge and skills is crucial for autonomous agents. Existing methods are typically based on either fixed-size models that struggle to learn a large number of diverse behaviors, or growing-size models that scale poorly with the number of tasks. In this work, we aim to strike a better balance between an agent's size and performance by designing a method that grows adaptively depending on the task sequence. We introduce Continual Subspace of Policies (CSP), a new approach that incrementally builds a subspace of policies for training a reinforcement learning agent on a sequence of tasks. The subspace's high expressivity allows CSP to perform well for many different tasks while growing sublinearly with the number of tasks. Our method does not suffer from forgetting and displays positive transfer to new tasks. CSP outperforms a number of popular baselines on a wide range of scenarios from two challenging domains, Brax (locomotion) and Continual World (manipulation).
Code (1)
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning
In dynamic domains such as autonomous robotics and video game simulations, agents must continuously adapt to new tasks while retaining previously acquired skills. This ongoing process, known as Continual Reinforcement Le…
Continual LearningMuJoCoreinforcement-learningReinforcement LearningShared LoRA Subspaces for almost Strict Continual Learning
Adapting large pretrained models to new tasks efficiently and continually is crucial for real-world deployment but remains challenging due to catastrophic forgetting and the high cost of retraining. While parameter-effic…
Natural Language UnderstandingText-to-Image GenerationImage ClassificationContinual LearningLANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning
On-device learning is essential for personalization, privacy, and long-term adaptation in resource-constrained environments. Achieving this requires efficient learning, both fine-tuning existing models and continually ac…
Continual LearningHierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models
Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting. Existing methods primarily restrict parameter updates but…
class-incremental learningContinual LearningThe Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation
Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data. Existing CTTA methods face an efficiency-generalization trade-…
Test-time Adaptation