End-to-end Keyword Spotting using Neural Architecture Search and Quantization
This paper introduces neural architecture search (NAS) for the automatic discovery of end-to-end keyword spotting (KWS) models in limited resource environments. We employ a differentiable NAS approach to optimize the structure of convolutional neural networks (CNNs) operating on raw audio waveforms. After a suitable KWS model is found with NAS, we conduct quantization of weights and activations to reduce the memory footprint. We conduct extensive experiments on the Google speech commands dataset. In particular, we compare our end-to-end approach to mel-frequency cepstral coefficient (MFCC) based systems. For quantization, we compare fixed bit-width quantization and trained bit-width quantization. Using NAS only, we were able to obtain a highly efficient model with an accuracy of 95.55% using 75.7k parameters and 13.6M operations. Using trained bit-width quantization, the same model achieves a test accuracy of 93.76% while using on average only 2.91 bits per activation and 2.51 bits per weight.
Code (0)
등록된 구현이 없습니다.
Tasks
Keyword SpottingNeural Architecture SearchQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Low-bit quantization and quantization-aware training for small-footprint keyword spotting
We investigate low-bit quantization to reduce computational cost of deep neural network (DNN) based keyword spotting (KWS). We propose approaches to further reduce quantization bits via integrating quantization into keyw…
Keyword SpottingQuantizationSmall-Footprint Keyword SpottingResource-efficient DNNs for Keyword Spotting using Neural Architecture Search and Quantization
This paper introduces neural architecture search (NAS) for the automatic discovery of small models for keyword spotting (KWS) in limited resource environments. We employ a differentiable NAS approach to optimize the stru…
Keyword SpottingNeural Architecture SearchQuantizationEncoder-Decoder Neural Architecture Optimization for Keyword Spotting
Keyword spotting aims to identify specific keyword audio utterances. In recent years, deep convolutional neural networks have been widely utilized in keyword spotting systems. However, their model architectures are mainl…
DecoderKeyword SpottingNeural Architecture SearchNeural Architecture Search For Keyword Spotting
Deep neural networks have recently become a popular solution to keyword spotting systems, which enable the control of smart devices via voice. In this paper, we apply neural architecture search to search for convolutiona…
Keyword SpottingNeural Architecture SearchBifocal Neural ASR: Exploiting Keyword Spotting for Inference Optimization
We present Bifocal RNN-T, a new variant of the Recurrent Neural Network Transducer (RNN-T) architecture designed for improved inference time latency on speech recognition tasks. The architecture enables a dynamic pivot f…
Inference OptimizationKeyword SpottingQuantizationspeech-recognition+1