Quantization in Spiking Neural Networks
In spiking neural networks (SNN), at each node, an incoming sequence of weighted Dirac pulses is converted into an output sequence of weighted Dirac pulses by a leaky-integrate-and-fire (LIF) neuron model based on spike aggregation and thresholding. We show that this mapping can be understood as a quantization operator and state a corresponding formula for the quantization error by means of the Alexiewicz norm. This analysis has implications for rethinking re-initialization in the LIF model, leading to the proposal of 'reset-to-mod' as a modulo-based reset variant.
Code (1)
Tasks
QuantizationSimilar Papers 제목 키워드 기반
Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search
Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating atten…
Model CompressionQuantizationTemporal-adaptive Weight Quantization for Spiking Neural Networks
Weight quantization in spiking neural networks (SNNs) could further reduce energy consumption. However, quantizing weights without sacrificing accuracy remains challenging. In this study, inspired by astrocyte-mediated s…
GT-SVQ: A Linear-Time Graph Transformer for Node Classification Using Spiking Vector Quantization
Graph Transformers (GTs), which simultaneously integrate message-passing and self-attention mechanisms, have achieved promising empirical results in some graph prediction tasks. Although these approaches show the potenti…
Graph LearningGraph Representation LearningNode ClassificationQuantization+1Exploring Extreme Quantization in Spiking Language Models
Despite the growing prevalence of large language model (LLM) architectures, a crucial concern persists regarding their energy and power consumption, which still lags far behind the remarkable energy efficiency of the hum…
Knowledge DistillationLanguage ModelingLanguage ModellingLarge Language Model+3MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networks
We propose Multiplier-less INTeger (MINT) quantization, a uniform quantization scheme that efficiently compresses weights and membrane potentials in spiking neural networks (SNNs). Unlike previous SNN quantization method…
Quantization