Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor
Deep learning has shown its human-level performance in various applications. However, current deep learning models are characterised by catastrophic forgetting of old knowledge when learning new classes. This poses a challenge particularly in intelligent diagnosis systems where initially only training data of a limited number of diseases are available. In this case, updating the intelligent system with data of new diseases would inevitably downgrade its performance on previously learned diseases. Inspired by the process of learning new knowledge in human brains, we propose a Bayesian generative model for continual learning built on a fixed pre-trained feature extractor. In this model, knowledge of each old class can be compactly represented by a collection of statistical distributions, e.g. with Gaussian mixture models, and naturally kept from forgetting in continual learning over time. Unlike existing class-incremental learning methods, the proposed approach is not sensitive to the continual learning process and can be additionally well applied to the data-incremental learning scenario. Experiments on multiple medical and natural image classification tasks showed that the proposed approach outperforms state-of-the-art approaches which even keep some images of old classes during continual learning of new classes.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningContinual Learningimage-classificationImage ClassificationIncremental LearningSimilar Papers 제목 키워드 기반
On Sequential Loss Approximation for Continual Learning
We introduce for continual learning Autodiff Quadratic Consolidation (AQC), which approximates the previous loss function with a quadratic function, and Neural Consolidation (NC), which approximates the previous loss fun…
class-incremental learningClass Incremental LearningContinual LearningIncremental Learning+1Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases
Currently intelligent diagnosis systems lack the ability of continually learning to diagnose new diseases once deployed, under the condition of preserving old disease knowledge. In particular, updating an intelligent dia…
Continual LearningOnline Continual Learning Without the Storage Constraint
Traditional online continual learning (OCL) research has primarily focused on mitigating catastrophic forgetting with fixed and limited storage allocation throughout an agent's lifetime. However, a broad range of real-wo…
Continual LearningAre Current Continual Learning Methods Truly Agnostic? Introducing OPRE, a Step Toward Agnostic Continual Learning
In order to achieve Continual Learning (CL), the problem of catastrophic forgetting, one that has plagued neural networks since their inception, must be overcome. The evaluation of continual learning methods relies on sp…
Continual LearningContinual Learning Using Pseudo-Replay via Latent Space Sampling
This paper investigates continual learning in the setting of class-incremental learning (CIL). Although numerous techniques have been proposed, CIL remains to be a highly challenging problem due to catastrophic forgettin…
class-incremental learningClass Incremental LearningContinual LearningIncremental Learning