TRGP: Trust Region Gradient Projection for Continual Learning

2022-02-07 · ICLR 2022 4 · Sen Lin, Li Yang, Deliang Fan, Junshan Zhang

Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2202.02931

Code (1)

LYang-666/TRGP 공식 구현 pytorch

Tasks

Continual Learning Transfer Learning

Trust Region-Guided Proximal Policy Optimization

2019-01-29 · NeurIPS 2019 12 · Yuhui Wang, Hao He, Xiaoyang Tan, Yaozhong Gan

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method,…

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Trust Region Continual Learning as an Implicit Meta-Learner

2026-02-02 · Zekun Wang, Anant Gupta, Christopher J. MacLellan arxiv

Continual learning aims to acquire tasks sequentially without catastrophic forgetting, yet standard strategies face a core tradeoff: regularization-based methods (e.g., EWC) can overconstrain updates when task optima are…

Continual Learning Image Generation

Boost Like a (Var)Pro: Trust-Region Gradient Boosting via Variable Projection

2026-03-24 · Abhijit Chowdhary, Elizabeth Newman, Deepanshu Verma arxiv

Gradient boosting, a method of building additive ensembles from weak learners, has established itself as a practical and theoretically-motivated approach to approximate functions, especially using decision tree weak lear…

Differentiable Trust Region Layers for Deep Reinforcement Learning

2021-01-22 · ICLR 2021 1 · Fabian Otto, Philipp Becker, Ngo Anh Vien, Hanna Carolin Ziesche 외

Trust region methods are a popular tool in reinforcement learning as they yield robust policy updates in continuous and discrete action spaces. However, enforcing such trust regions in deep reinforcement learning is diff…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Continual Learning with Scaled Gradient Projection

2023-02-02 · Gobinda Saha, Kaushik Roy

In neural networks, continual learning results in gradient interference among sequential tasks, leading to catastrophic forgetting of old tasks while learning new ones. This issue is addressed in recent methods by storin…

Continual Learning image-classification Image Classification