paper-with-me

홈 › Papers

Heterogeneous quantization regularizes spiking neural network activity

2024-09-27 · Roy Moyal, Kyrus R. Mama, Matthew Einhorn, Ayon Borthakur, Thomas A. Cleland

The learning and recognition of object features from unregulated input has been a longstanding challenge for artificial intelligence systems. Brains are adept at learning stable representations given small samples of noisy observations; across sensory modalities, this capacity is aided by a cascade of signal conditioning steps informed by domain knowledge. The olfactory system, in particular, solves a source separation and denoising problem compounded by concentration variability, environmental interference, and unpredictably correlated sensor affinities. To function optimally, its plastic network requires statistically well-behaved input. We present a data-blind neuromorphic signal conditioning strategy whereby analog data are normalized and quantized into spike phase representations. Input is delivered to a column of duplicated spiking principal neurons via heterogeneous synaptic weights; this regularizes layer utilization, yoking total activity to the network's operating range and rendering internal representations robust to uncontrolled open-set stimulus variance. We extend this mechanism by adding a data-aware calibration step whereby the range and density of the quantization weights adapt to accumulated input statistics, optimizing resource utilization by balancing activity regularization and information retention.

📄 PDF Abstract BibTeX arXiv:2409.18396

Code (0)

등록된 구현이 없습니다.

Tasks

DenoisingQuantization

Similar Papers 제목 키워드 기반

Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search

2024-12-07 · Boxun Xu, Yufei Song, Peng Li

Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating atten…

Model CompressionQuantization

Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles

2023-02-22 · Biswadeep Chakraborty, Saibal Mukhopadhyay

This paper shows that the heterogeneity in neuronal and synaptic dynamics reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while improving prediction performance, enabling spike-efficient (unsupe…

Bayesian OptimizationDiversityLearning TheoryPrediction+3

Exploring Extreme Quantization in Spiking Language Models

2024-05-04 · Malyaban Bal, Yi Jiang, Abhronil Sengupta

Despite the growing prevalence of large language model (LLM) architectures, a crucial concern persists regarding their energy and power consumption, which still lags far behind the remarkable energy efficiency of the hum…

Knowledge DistillationLanguage ModelingLanguage ModellingLarge Language Model+3

Learning recurrent dynamics in spiking networks

2018-03-18 · Christopher Kim, Carson Chow

Spiking activity of neurons engaged in learning and performing a task show complex spatiotemporal dynamics. While the output of recurrent network models can learn to perform various tasks, the possible range of recurrent…

Neural Data Transformer 2: Multi-context Pretraining for Neural Spiking Activity

2023-09-21 · NeurIPS 2023 11

The neural population spiking activity recorded by intracortical brain-computer interfaces (iBCIs) contain rich structure. Current models of such spiking activity are largely prepared for individual experimental contexts…