Stochasticity and Robustness in Spiking Neural Networks
Artificial neural networks normally require precise weights to operate, despite their origins in biological systems, which can be highly variable and noisy. When implementing artificial networks which utilize analog 'synaptic' devices to encode weights, however, inherent limits are placed on the accuracy and precision with which these values can be encoded. In this work, we investigate the effects that inaccurate synapses have on spiking neurons and spiking neural networks. Starting with a mathematical analysis of integrate-and-fire (IF) neurons, including different non-idealities (such as leakage and channel noise), we demonstrate that noise can be used to make the behavior of IF neurons more robust to synaptic inaccuracy. We then train spiking networks which utilize IF neurons with and without noise and leakage, and experimentally confirm that the noisy networks are more robust. Lastly, we show that a noisy network can tolerate the inaccuracy expected when hafnium-oxide based resistive random-access memory is used to encode synaptic weights.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Intelligence plays dice: Stochasticity is essential for machine learning
Many fields view stochasticity as a way to gain computational efficiency, while often having to trade off accuracy. In this perspective article, we argue that stochasticity plays a fundamentally different role in machine…
BIG-bench Machine LearningComputational EfficiencyStochastic Spiking Neuron Based SNN Can be Inherently Bayesian
Uncertainty in biological neural systems appears to be computationally beneficial rather than detrimental. However, in neuromorphic computing systems, device variability often limits performance, including accuracy and e…
Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars
Spiking Neural Networks (SNNs) have recently emerged as the low-power alternative to Artificial Neural Networks (ANNs) owing to their asynchronous, sparse, and binary information processing. To improve the energy-efficie…
Stochastic IMT (insulator-metal-transition) neurons: An interplay of thermal and threshold noise at bifurcation
Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Electronic implementation of such stochastic networks is currently limited to addition of a…
Spiking Analog VLSI Neuron Assemblies as Constraint Satisfaction Problem Solvers
Solving constraint satisfaction problems (CSPs) is a notoriously expensive computational task. Recently, it has been proposed that efficient stochastic solvers can be obtained through appropriately configured spiking neu…