Continual Learning Through Synaptic Intelligence
While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many tasks simultaneously. In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. Each synapse accumulates task relevant information over time, and exploits this information to rapidly store new memories without forgetting old ones. We evaluate our approach on continual learning of classification tasks, and show that it dramatically reduces forgetting while maintaining computational efficiency.
Code (6)
Tasks
Computational EfficiencyContinual LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
Theories of synaptic memory consolidation and intelligent plasticity for continual learning
Humans and animals learn throughout life. Such continual learning is crucial for intelligence. In this chapter, we examine the pivotal role plasticity mechanisms with complex internal synaptic dynamics could play in enab…
Continual LearningOn-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling
Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunn…
Continual LearningSPICED: A Synaptic Homeostasis-Inspired Framework for Unsupervised Continual EEG Decoding
Human brain achieves dynamic stability-plasticity balance through synaptic homeostasis. Inspired by this biological principle, we propose SPICED: a neuromorphic framework that integrates the synaptic homeostasis mechanis…
Continual LearningEeg DecodingRehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligence
Continual Federated Learning (CFL) allows distributed devices to collaboratively learn novel concepts from continuously shifting training data while avoiding knowledge forgetting of previously seen tasks. To tackle this …
Federated LearningNovel ConceptsIncorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world. For this purpose, a desirable solution should properly balance memory stability with learning plasticity, and ac…
Continual Learning