Structured Sparsity and Weight-adaptive Pruning for Memory and Compute efficient Whisper models
Whisper models have achieved remarkable progress in speech recognition; yet their large size remains a bottleneck for deployment on resource-constrained edge devices. This paper proposes a framework to design fine-tuned variants of Whisper which address the above problem. Structured sparsity is enforced via the Sparse Group LASSO penalty as a loss regularizer, to reduce the number of FLOating Point operations (FLOPs). Further, a weight statistics aware pruning algorithm is proposed. We also design our custom text normalizer for WER evaluation. On Common Voice 11.0 Hindi dataset, we obtain, without degrading WER, (a) 35.4% reduction in model parameters, 14.25% lower memory consumption and 18.5% fewer FLOPs on Whisper-small, and (b) 31% reduction in model parameters, 15.29% lower memory consumption and 16.95% fewer FLOPs on Whisper-medium; and, (c) substantially outperform the state-of-the-art Iterative Magnitude Pruning based method by pruning 18.7% more parameters along with a 12.31 reduction in WER.
Code (0)
등록된 구현이 없습니다.
Tasks
Speech RecognitionResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #4 | Speech Recognition | Common Voice | Structured Sparsity and Weight-adaptive | Test WER: 11.0 |
Similar Papers 제목 키워드 기반
Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
This paper presents Thanos, a novel weight-pruning algorithm designed to reduce the memory footprint and enhance the computational efficiency of large language models (LLMs) by removing redundant weights while maintainin…
Computational EfficiencyLanguage ModelingLanguage ModellingLarge Language Model+1DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs
Large language models (LLMs) deliver strong performance but are difficult to deploy due to high memory and compute costs. While pruning reduces these demands, most methods ignore activation sparsity observed at runtime. …
GPUStructPrune: Structured Global Pruning asymptotics with $\mathcal{O}(\sqrt{N})$ GPU Memory
Pruning is critical for scaling large language models (LLMs). Global pruning achieves strong performance but requires $\mathcal{O}(N)$ memory, which is infeasible for billion-parameter models. Local pruning reduces GPU m…
Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models
The rapid advancement in Large Language Models (LLMs) has markedly enhanced the capabilities of language understanding and generation. However, the substantial model size poses hardware challenges, affecting both memory …
Computational EfficiencyModel CompressionNetwork PruningDynamic Probabilistic Pruning: Training sparse networks based on stochastic and dynamic masking
Deep Learning (DL) models are known to be heavily over-parametrized, resulting in a large memory footprint and power consumption. This hampers the use of such models in hardware-constrained edge technologies such as wear…
Model Compression