Compute-in-Memory Implementation of State Space Models for Event Sequence Processing
State space models (SSMs) have recently emerged as a powerful framework for long sequence processing, outperforming traditional methods on diverse benchmarks. Fundamentally, SSMs can generalize both recurrent and convolutional networks and have been shown to even capture key functions of biological systems. Here we report an approach to implement SSMs in energy-efficient compute-in-memory (CIM) hardware to achieve real-time, event-driven processing. Our work re-parameterizes the model to function with real-valued coefficients and shared decay constants, reducing the complexity of model mapping onto practical hardware systems. By leveraging device dynamics and diagonalized state transition parameters, the state evolution can be natively implemented in crossbar-based CIM systems combined with memristors exhibiting short-term memory effects. Through this algorithm and hardware co-design, we show the proposed system offers both high accuracy and high energy efficiency while supporting fully asynchronous processing for event-based vision and audio tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Event-based visionSimilar Papers 제목 키워드 기반
Memory Efficient Corner Detection for Event-driven Dynamic Vision Sensors
Event cameras offer low-latency and data compression for visual applications, through event-driven operation, that can be exploited for edge processing in tiny autonomous agents. Robust, accurate and low latency extracti…
Data CompressionEdge-computingSynapse Compression for Event-Based Convolutional-Neural-Network Accelerators
Manufacturing-viable neuromorphic chips require novel computer architectures to achieve the massively parallel and efficient information processing the brain supports so effortlessly. Emerging event-based architectures a…
IMSSA: Deploying modern state-space models on memristive in-memory compute hardware
Processing long temporal sequences is a key challenge in deep learning. In recent years, Transformers have become state-of-the-art for this task, but suffer from excessive memory requirements due to the need to explicitl…
GPUQuantizationState Space ModelsTemporal SequencesA Machine Learning Approach to Predicting Single Event Upsets
A single event upset (SEU) is a critical soft error that occurs in semiconductor devices on exposure to ionising particles from space environments. SEUs cause bit flips in the memory component of semiconductors. This cre…
Hardware Synthesis of State-Space Equations; Application to FPGA Implementation of Shallow and Deep Neural Networks
Nowadays, shallow and deep Neural Networks (NNs) have vast applications including biomedical engineering, image processing, computer vision, and speech recognition. Many researchers have developed hardware accelerators i…
speech-recognitionSpeech RecognitionState Space Models