Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning
In dynamic domains such as autonomous robotics and video game simulations, agents must continuously adapt to new tasks while retaining previously acquired skills. This ongoing process, known as Continual Reinforcement Learning, presents significant challenges, including the risk of forgetting past knowledge and the need for scalable solutions as the number of tasks increases. To address these issues, we introduce HIerarchical LOW-rank Subspaces of Policies (HILOW), a novel framework designed for continual learning in offline navigation settings. HILOW leverages hierarchical policy subspaces to enable flexible and efficient adaptation to new tasks while preserving existing knowledge. We demonstrate, through a careful experimental study, the effectiveness of our method in both classical MuJoCo maze environments and complex video game-like simulations, showcasing competitive performance and satisfying adaptability according to classical continual learning metrics, in particular regarding memory usage. Our work provides a promising framework for real-world applications where continuous learning from pre-collected data is essential.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningMuJoCoreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Offline Hierarchical Reinforcement Learning via Inverse Optimization
Hierarchical policies enable strong performance in many sequential decision-making problems, such as those with high-dimensional action spaces, those requiring long-horizon planning, and settings with sparse rewards. How…
Decision MakingHierarchical Reinforcement Learningreinforcement-learningReinforcement Learning+2Efficient Hierarchical Implicit Flow Q-learning for Offline Goal-conditioned Reinforcement Learning
Offline goal-conditioned reinforcement learning (GCRL) is a practical reinforcement learning paradigm that aims to learn goal-conditioned policies from reward-free offline data. Despite recent advances in hierarchical ar…
Reinforcement LearningContinual Task Learning through Adaptive Policy Self-Composition
Training a generalizable agent to continually learn a sequence of tasks from offline trajectories is a natural requirement for long-lived agents, yet remains a significant challenge for current offline reinforcement lear…
Continual LearningOffline RLReinforcement Learning (RL)Hierarchical Continual Reinforcement Learning via Large Language Model
The ability to learn continuously in dynamic environments is a crucial requirement for reinforcement learning (RL) agents applying in the real world. Despite the progress in continual reinforcement learning (CRL), existi…
Language ModelingLanguage ModellingLarge Language Modelmodel+4Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery
Offline reinforcement learning (RL) enables the agent to effectively learn from logged data, which significantly extends the applicability of RL algorithms in real-world scenarios where exploration can be expensive or un…
D4RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)