Neuromimetic metaplasticity for adaptive continual learning
Conventional intelligent systems based on deep neural network (DNN) models encounter challenges in achieving human-like continual learning due to catastrophic forgetting. Here, we propose a metaplasticity model inspired by human working memory, enabling DNNs to perform catastrophic forgetting-free continual learning without any pre- or post-processing. A key aspect of our approach involves implementing distinct types of synapses from stable to flexible, and randomly intermixing them to train synaptic connections with different degrees of flexibility. This strategy allowed the network to successfully learn a continuous stream of information, even under unexpected changes in input length. The model achieved a balanced tradeoff between memory capacity and performance without requiring additional training or structural modifications, dynamically allocating memory resources to retain both old and new information. Furthermore, the model demonstrated robustness against data poisoning attacks by selectively filtering out erroneous memories, leveraging the Hebb repetition effect to reinforce the retention of significant data.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningData PoisoningSimilar Papers 제목 키워드 기반
Probabilistic Metaplasticity for Continual Learning with Memristors
Edge devices operating in dynamic environments critically need the ability to continually learn without catastrophic forgetting. The strict resource constraints in these devices pose a major challenge to achieve this, as…
Continual LearningBayesian Metaplasticity from Synaptic Uncertainty
Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (MESU), inspired by metaplasticity and Baye…
Bayesian InferenceContinual LearningLifelong learningPermuted-MNISTGenesis: A Spiking Neuromorphic Accelerator With On-chip Continual Learning
Continual learning, the ability to acquire and transfer knowledge through a models lifetime, is critical for artificial agents that interact in real-world environments. Biological brains inherently demonstrate these capa…
Continual LearningStatistical mechanics of continual learning: variational principle and mean-field potential
An obstacle to artificial general intelligence is set by continual learning of multiple tasks of different nature. Recently, various heuristic tricks, both from machine learning and from neuroscience angles, were propose…
Continual LearningMulti-Task LearningBayesian continual learning and forgetting in neural networks
Biological synapses effortlessly balance memory retention and flexibility, yet artificial neural networks still struggle with the extremes of catastrophic forgetting and catastrophic remembering. Here, we introduce Metap…
Bayesian InferenceContinual Learningimage-classificationImage Classification+2