paper-with-me

홈 › Papers

Long Short-Term Memory Spiking Networks and Their Applications

2020-07-09 · Ali Lotfi Rezaabad, Sriram Vishwanath

Recent advances in event-based neuromorphic systems have resulted in significant interest in the use and development of spiking neural networks (SNNs). However, the non-differentiable nature of spiking neurons makes SNNs incompatible with conventional backpropagation techniques. In spite of the significant progress made in training conventional deep neural networks (DNNs), training methods for SNNs still remain relatively poorly understood. In this paper, we present a novel framework for training recurrent SNNs. Analogous to the benefits presented by recurrent neural networks (RNNs) in learning time series models within DNNs, we develop SNNs based on long short-term memory (LSTM) networks. We show that LSTM spiking networks learn the timing of the spikes and temporal dependencies. We also develop a methodology for error backpropagation within LSTM-based SNNs. The developed architecture and method for backpropagation within LSTM-based SNNs enable them to learn long-term dependencies with comparable results to conventional LSTMs.

📄 PDF Abstract BibTeX arXiv:2007.04779

Code (1)

AliLotfi92/SNNLSTM 공식 구현

Tasks

Time SeriesTime Series Analysis

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음
Tanh Activation 설명 없음
LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Similar Papers 제목 키워드 기반

Long Short-term Memory with Two-Compartment Spiking Neuron

2023-07-14 · Shimin Zhang, Qu Yang, Chenxiang Ma, Jibin Wu 외

The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. As a result, it remains a challenging task fo…

Volatile Memory Motifs: Minimal Spiking Neural Networks

2023-03-21 · Fabio Schittler Neves, Marc Timme

How spiking neuronal networks encode memories in their different time and spatial scales constitute a fundamental topic in neuroscience and neuro-inspired engineering. Much attention has been paid to large networks and l…

Short-Term Memory Through Persistent Activity: Evolution of Self-Stopping and Self-Sustaining Activity in Spiking Neural Networks

2014-11-25 · Julien Hubert, Takashi Ikegami

Memories in the brain are separated in two categories: short-term and long-term memories. Long-term memories remain for a lifetime, while short-term ones exist from a few milliseconds to a few minutes. Within short-term …

Expanding memory in recurrent spiking networks

2023-10-29 · Ismael Balafrej, Fabien Alibart, Jean Rouat

Recurrent spiking neural networks (RSNNs) are notoriously difficult to train because of the vanishing gradient problem that is enhanced by the binary nature of the spikes. In this paper, we review the ability of the curr…

Memory-Efficient Reversible Spiking Neural Networks

2023-12-13 · Hong Zhang, Yu Zhang

Spiking neural networks (SNNs) are potential competitors to artificial neural networks (ANNs) due to their high energy-efficiency on neuromorphic hardware. However, SNNs are unfolded over simulation time steps during the…

GPU