A Fixed Version of Quadratic Program in Gradient Episodic Memory
Gradient Episodic Memory is indeed a novel method for continual learning, which solves new problems quickly without forgetting previously acquired knowledge. However, in the process of studying the paper, we found there were some problems in the proof of the dual problem of Quadratic Program, so here we give our fixed version for this problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Revisiting Gradient Episodic Memory for Continual Learning
Gradient Episodic Memory (GEM) is an effective model for continual learning, where each gradient update for the current task is formulated as a quadratic program problem with inequality constraints that alleviate catastr…
Continual LearningTransfer LearningJoint learning of constraint weights and gradient inputs in Gradient Symbolic Computation with constrained optimization
This paper proposes a method for the joint optimization of constraint weights and symbol activations within the Gradient Symbolic Computation (GSC) framework. The set of grammars representable in GSC is proven to be a su…
Nonconvex Continual Learning with Episodic Memory
Continual learning aims to prevent catastrophic forgetting while learning a new task without accessing data of previously learned tasks. The memory for such learning scenarios build a small subset of the data for previo…
Continual Learningimage-classificationImage ClassificationA Sample-Efficient Algorithm for Episodic Finite-Horizon MDP with Constraints
Constrained Markov Decision Processes (CMDPs) formalize sequential decision-making problems whose objective is to minimize a cost function while satisfying constraints on various cost functions. In this paper, we conside…
Decision MakingSequential Decision MakingGEM-Style Constraints for PEFT with Dual Gradient Projection in LoRA
Full fine-tuning of Large Language Models (LLMs) is computationally costly, motivating Continual Learning (CL) approaches that utilize parameter-efficient adapters. We revisit Gradient Episodic Memory (GEM) within the Lo…
Continual Learning