A Proposal for Networks Capable of Continual Learning
We analyze the ability of computational units to retain past responses after parameter updates, a key property for system-wide continual learning. Neural networks trained with gradient descent lack this capability, prompting us to propose Modelleyen, an alternative approach with inherent response preservation. We demonstrate through experiments on modeling the dynamics of a simple environment and on MNIST that, despite increased computational complexity and some representational limitations at its current stage, Modelleyen achieves continual learning without relying on sample replay or predefined task boundaries.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Class-wise Classifier Design Capable of Continual Learning using Adaptive Resonance Theory-based Topological Clustering
This paper proposes a supervised classification algorithm capable of continual learning by utilizing an Adaptive Resonance Theory (ART)-based growing self-organizing clustering algorithm. The ART-based clustering algorit…
ClassificationClusteringContinual LearningContrast R-CNN for Continual Learning in Object Detection
The continual learning problem has been widely studied in image classification, while rare work has been explored in object detection. Some recent works apply knowledge distillation to constrain the model to retain old k…
Continual Learningimage-classificationImage ClassificationKnowledge Distillation+3Don't forget, there is more than forgetting: new metrics for Continual Learning
Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of conse…
AttributeComputational EfficiencyContinual LearningTransfer LearningToward industrial use of continual learning : new metrics proposal for class incremental learning
In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accur…
class-incremental learningClass Incremental LearningContinual LearningIncremental LearningBilevel Continual Learning
Continual learning (CL) studies the problem of learning a sequence of tasks, one at a time, such that the learning of each new task does not lead to the deterioration in performance on the previously seen ones while expl…
Bilevel OptimizationContinual LearningMeta-Learning