Distributed Compressed Sparse Row Format for Spiking Neural Network Simulation, Serialization, and Interoperability
With the increasing development of neuromorphic platforms and their related software tools as well as the increasing scale of spiking neural network (SNN) models, there is a pressure for interoperable and scalable representations of network state. In response to this, we discuss a parallel extension of a widely used format for efficiently representing sparse matrices, the compressed sparse row (CSR), in the context of supporting the simulation and serialization of large-scale SNNs. Sparse matrices for graph adjacency structure provide a natural fit for describing the connectivity of an SNN, and prior work in the area of parallel graph partitioning has developed the distributed CSR (dCSR) format for storing and ingesting large graphs. We contend that organizing additional network information, such as neuron and synapse state, in alignment with its adjacency as dCSR provides a straightforward partition-based distribution of network state. For large-scale simulations, this means each parallel process is only responsible for its own partition of state, which becomes especially useful when the size of an SNN exceeds the memory resources of a single compute node. For potentially long-running simulations, this also enables network serialization to and from disk (e.g. for checkpoint/restart fault-tolerant computing) to be performed largely independently between parallel processes. We also provide a potential implementation, and put it forward for adoption within the neural computing community.
Code (0)
등록된 구현이 없습니다.
Tasks
graph partitioningNeural Network simulationSimilar Papers 제목 키워드 기반
Sparse Diffusion Steepest-Descent for One Bit Compressed Sensing in Wireless Sensor Networks
This letter proposes a sparse diffusion steepest-descent algorithm for one bit compressed sensing in wireless sensor networks. The approach exploits the diffusion strategy from distributed learning in the one bit compres…
compressed sensingSparse Compressed Spiking Neural Network Accelerator for Object Detection
Spiking neural networks (SNNs), which are inspired by the human brain, have recently gained popularity due to their relatively simple and low-power hardware for transmitting binary spikes and highly sparse activation map…
Objectobject-detectionObject DetectionSparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks
Spiking Neural Networks (SNNs) are biologically-inspired models that are capable of processing information in streams of action potentials. However, simulating and training SNNs is computationally expensive due to the ne…
A Novel Approximate Hamming Weight Computing for Spiking Neural Networks: an FPGA Friendly Architecture
Hamming weights of sparse and long binary vectors are important modules in many scientific applications, particularly in spiking neural networks that are of our interest. To improve both area and latency of their FPGA im…
Spatio-Temporal Pruning for Compressed Spiking Large Language Models
Large Language Models (LLMs) present significant challenges for deployment in energy-constrained environments due to their large model sizes and high inference latency. Spiking Neural Networks (SNNs), inspired by the spa…
Computational EfficiencyKnowledge Distillation