Rethinking Spiking Neural Networks from an Ensemble Learning Perspective

2025-02-20 · Yongqi Ding, Lin Zuo, Mengmeng Jing, Pei He, Hanpu Deng

Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal membrane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mitigating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in 1D speech, 2D object, and 3D point cloud recognition tasks. In particular, on the challenging CIFAR10-DVS dataset, we achieved 83.20\% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.

📄 PDF Abstract BibTeX arXiv:2502.14218

Code (0)

등록된 구현이 없습니다.

Tasks

Ensemble Learning

Rethinking LLM Ensembling from the Perspective of Mixture Models

2026-05-01 · Jiale Fu, Yuchu Jiang, Peijun Wu, Chonghan Liu 외 arxiv

Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multiple models and selecting the most probab…

Ensembles of Spiking Neural Networks

2020-10-15 · Georgiana Neculae, Oliver Rhodes, Gavin Brown

This paper demonstrates how to construct ensembles of spiking neural networks producing state-of-the-art results, achieving classification accuracies of 98.71%, 100.0%, and 99.09%, on the MNIST, NMNIST and DVS Gesture da…

Diversity Ensemble Learning

Rethinking Spiking Self-Attention Mechanism: Implementing a-XNOR Similarity Calculation in Spiking Transformers

2025-01-01 · CVPR 2025 1 · Yichen Xiao, Shuai Wang, Dehao Zhang, Wenjie Wei 외

Transformers significantly raise the performance limits across various tasks, spurring research into integrating them into spiking neural networks. However, a notable performance gap remains between existing spiking …

Attribute

Quantization in Spiking Neural Networks

2023-05-13 · Bernhard A. Moser, Michael Lunglmayr

In spiking neural networks (SNN), at each node, an incoming sequence of weighted Dirac pulses is converted into an output sequence of weighted Dirac pulses by a leaky-integrate-and-fire (LIF) neuron model based on spike …

Quantization

Ensemble plasticity and network adaptability in SNNs

2022-03-11 · Mahima Milinda Alwis Weerasinghe, David Parry, Grace Wang, Jacqueline Whalley

Artificial Spiking Neural Networks (ASNNs) promise greater information processing efficiency because of discrete event-based (i.e., spike) computation. Several Machine Learning (ML) applications use biologically inspired…

EEG Electroencephalogram (EEG)Ensemble Learning