On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling
Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunneling device can implement synaptic memory consolidation similar to what can be achieved by algorithmic consolidation models like the cascade and the elastic weight consolidation (EWC) models. The proposed FN-synapse not only stores the synaptic weight but also stores the synapse's historical usage statistic on the device itself. We also show that the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and we demonstrate that a network comprising FN-synapses outperforms a comparable EWC network for a small benchmark continual learning task. With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and persistent learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
An Adaptive Synaptic Array using Fowler-Nordheim Dynamic Analog Memory
In this paper we present a synaptic array that uses dynamical states to implement an analog memory for energy-efficient training of machine learning (ML) systems. Each of the analog memory elements is a micro-dynamical s…
Higher-Order Neuromorphic Ising Machines -- Autoencoders and Fowler-Nordheim Annealers are all you need for Scalability
We report a higher-order neuromorphic Ising machine that exhibits superior scalability compared to architectures based on quadratization, while also achieving state-of-the-art quality and reliability in solutions with co…
AllCombinatorial OptimizationON-OFF Neuromorphic ISING Machines using Fowler-Nordheim Annealers
We introduce NeuroSA, a neuromorphic architecture specifically designed to ensure asymptotic convergence to the ground state of an Ising problem using a Fowler-Nordheim quantum mechanical tunneling based threshold-anneal…
Combinatorial OptimizationSYNERgy between SYNaptic consolidation and Experience Replay for general continual learning
Continual learning (CL) in the brain is facilitated by a complex set of mechanisms. This includes the interplay of multiple memory systems for consolidating information as posited by the complementary learning systems (C…
Continual LearningOptimal stimulation protocol in a bistable synaptic consolidation model
Consolidation of synaptic changes in response to neural activity is thought to be fundamental for memory maintenance over a timescale of hours. In experiments, synaptic consolidation can be induced by repeatedly stimulat…