Efficient Implementation of a Multi-Layer Gradient-Free Online-Trainable Spiking Neural Network on FPGA
This paper presents an efficient hardware implementation of the recently proposed Optimized Deep Event-driven Spiking Neural Network Architecture (ODESA). ODESA is the first network to have end-to-end multi-layer online local supervised training without using gradients and has the combined adaptation of weights and thresholds in an efficient hierarchical structure. This research shows that the network architecture and the online training of weights and thresholds can be implemented efficiently on a large scale in hardware. The implementation consists of a multi-layer Spiking Neural Network (SNN) and individual training modules for each layer that enable online self-learning without using back-propagation. By using simple local adaptive selection thresholds, a Winner-Takes-All (WTA) constraint on each layer, and a modified weight update rule that is more amenable to hardware, the trainer module allocates neuronal resources optimally at each layer without having to pass high-precision error measurements across layers. All elements in the system, including the training module, interact using event-based binary spikes. The hardware-optimized implementation is shown to preserve the performance of the original algorithm across multiple spatial-temporal classification problems with significantly reduced hardware requirements.
Code (0)
등록된 구현이 없습니다.
Tasks
Self-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FPGA Implementation of Multi-Layer Machine Learning Equalizer with On-Chip Training
We design and implement an adaptive machine learning equalizer that alternates multiple linear and nonlinear computational layers on an FPGA. On-chip training via gradient backpropagation is shown to allow for real-time …
Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games
Multi-agent games in dynamic nonlinear settings are challenging due to the time-varying interactions among the agents and the non-stationarity of the (potential) Nash equilibria. In this paper we consider model-free game…
Gradient Diffusion: Enhancing Existing Neural Models with Homeostatic Control and Tuning
Realistic brain modeling requires precise estimation of numerous unobserved parameters, a task hindered by complex nonlinearities and the inaccessibility of the brain's full dynamical state. Current multicompartmental-mo…
BP(λ): Online Learning via Synthetic Gradients
Training recurrent neural networks typically relies on backpropagation through time (BPTT). BPTT depends on forward and backward passes to be completed, rendering the network locked to these computations before loss grad…
Reinforcement Learning (RL)DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule
We propose a tuning-free dynamic SGD step size formula, which we call Distance over Gradients (DoG). The DoG step sizes depend on simple empirical quantities (distance from the initial point and norms of gradients) and h…
Transfer Learning