Variational Continual Learning
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.
Code (8)
Tasks
Continual LearningVariational InferenceSimilar Papers 제목 키워드 기반
Adaptive Variational Continual Learning via Task-Heuristic Modelling
Variational continual learning (VCL) is a turn-key learning algorithm that has state-of-the-art performance among the best continual learning models. In our work, we explore an extension of the generalized variational co…
Continual LearningModel OptimizationEVCL: Elastic Variational Continual Learning with Weight Consolidation
Continual learning aims to allow models to learn new tasks without forgetting what has been learned before. This work introduces Elastic Variational Continual Learning with Weight Consolidation (EVCL), a novel hybrid mod…
Continual LearningIncremental LearningImproving and Understanding Variational Continual Learning
In the continual learning setting, tasks are encountered sequentially. The goal is to learn whilst i) avoiding catastrophic forgetting, ii) efficiently using model capacity, and iii) employing forward and backward transf…
Continual LearningPermuted-MNISTSplit-MNISTTransfer LearningKernel Continual Learning
This paper introduces kernel continual learning, a simple but effective variant of continual learning that leverages the non-parametric nature of kernel methods to tackle catastrophic forgetting. We deploy an episodic me…
Continual LearningVariational InferenceVariational Auto-Regressive Gaussian Processes for Continual Learning
Through sequential construction of posteriors on observing data online, Bayes' theorem provides a natural framework for continual learning. We develop Variational Auto-Regressive Gaussian Processes (VAR-GPs), a principle…
Bayesian InferenceContinual LearningGaussian Processes