QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models
Structured State Space models (SSM) have recently emerged as a new class of deep learning models, particularly well-suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of Transformers, makes them attractive candidates for deployment on resource-constrained edge-computing devices. While recent works have explored the effect of quantization-aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in-memory computing (AIMC) chips. In this work, we demonstrate that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. We analyze the relation between model size and numerical precision, and show that QAT enhances robustness to analog noise and enables structural pruning. Finally, we integrate these techniques to deploy SSMs on a memristive analog in-memory computing substrate and highlight the resulting benefits in terms of computational efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencyEdge-computingQuantizationState Space ModelsSimilar Papers 제목 키워드 기반
Deep Microcompression: Structured Pruning and Bit-packed Quantization for Microcontrollers
This paper introduces Deep Microcompression (DMC), a hardware-aware pipeline for deep learning inference on bare-metal microcontrollers. DMC integrates structured pruning, quantization-aware training, and fixed-length bi…
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization
The transformer extends its success from the language to the vision domain. Because of the stacked self-attention and cross-attention blocks, the acceleration deployment of vision transformer on GPU hardware is challengi…
BenchmarkingGPUKnowledge DistillationQuantizationIMSSA: Deploying modern state-space models on memristive in-memory compute hardware
Processing long temporal sequences is a key challenge in deep learning. In recent years, Transformers have become state-of-the-art for this task, but suffer from excessive memory requirements due to the need to explicitl…
GPUQuantizationState Space ModelsTemporal SequencesFrom Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
Deploying models, especially large language models (LLMs), is becoming increasingly attractive to a broader user base, including those without specialized expertise. However, due to the resource constraints of certain ha…
Efficient Network Inference via Hardware-Aware Architecture Search, Model Pruning & Quantization
Embedded global navigation satellite system (GNSS) interference monitoring requires fast and memory-efficient inference to process large volumes of raw in-phase and quadrature (IQ) samples in real time. At the same time,…
Neural Architecture SearchModel Compression