Low-bit quantization and quantization-aware training for small-footprint keyword spotting
We investigate low-bit quantization to reduce computational cost of deep neural network (DNN) based keyword spotting (KWS). We propose approaches to further reduce quantization bits via integrating quantization into keyword spotting model training, which we refer to as quantization-aware training. Our experimental results on large dataset indicate that quantization-aware training can recover performance models quantized to lower bits representations. By combining quantization-aware training and weight matrix factorization, we are able to significantly reduce model size and computation for small-footprint keyword spotting, while maintaining performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Keyword SpottingQuantizationSmall-Footprint Keyword SpottingSimilar Papers 제목 키워드 기반
Quantizing Small-Scale State-Space Models for Edge AI
State-space models (SSMs) have recently gained attention in deep learning for their ability to efficiently model long-range dependencies, making them promising candidates for edge-AI applications. In this paper, we analy…
QuantizationState Space ModelsOWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
Large language models (LLMs) with hundreds of billions of parameters require powerful server-grade GPUs for inference, limiting their practical deployment. To address this challenge, we introduce the outlier-aware weight…
parameter-efficient fine-tuningQuantizationSQuAT: Sharpness- and Quantization-Aware Training for BERT
Quantization is an effective technique to reduce memory footprint, inference latency, and power consumption of deep learning models. However, existing quantization methods suffer from accuracy degradation compared to ful…
QuantizationBitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks
Recently proposed methods for 1-bit and 1.58-bit quantization aware training investigate the performance and behavior of these methods in the context of large language models, finding state-of-the-art performance for mod…
QuantizationRoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
Low-Rank Adaptation (LoRA), as a representative Parameter-Efficient Fine-Tuning (PEFT)method, significantly enhances the training efficiency by updating only a small portion of the weights in Large Language Models (LLMs)…
parameter-efficient fine-tuningQuantization