Resource-efficient DNNs for Keyword Spotting using Neural Architecture Search and Quantization
This paper introduces neural architecture search (NAS) for the automatic discovery of small models for keyword spotting (KWS) in limited resource environments. We employ a differentiable NAS approach to optimize the structure of convolutional neural networks (CNNs) to maximize the classification accuracy while minimizing the number of operations per inference. Using NAS only, we were able to obtain a highly efficient model with 95.4% accuracy on the Google speech commands dataset with 494.8 kB of memory usage and 19.6 million operations. Additionally, weight quantization is used to reduce the memory consumption even further. We show that weight quantization to low bit-widths (e.g. 1 bit) can be used without substantial loss in accuracy. By increasing the number of input features from 10 MFCC to 20 MFCC we were able to increase the accuracy to 96.3% at 340.1 kB of memory usage and 27.1 million operations.
Code (2)
Tasks
Keyword SpottingNeural Architecture SearchQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Data Aware Differentiable Neural Architecture Search for Tiny Keyword Spotting Applications
The success of Machine Learning is increasingly tempered by its significant resource footprint, driving interest in efficient paradigms like TinyML. However, the inherent complexity of designing TinyML systems hampers th…
Neural Architecture SearchKeyword SpottingEncoder-Decoder Neural Architecture Optimization for Keyword Spotting
Keyword spotting aims to identify specific keyword audio utterances. In recent years, deep convolutional neural networks have been widely utilized in keyword spotting systems. However, their model architectures are mainl…
DecoderKeyword SpottingNeural Architecture SearchOn-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation
Large self-supervised models are effective feature extractors, but their application is challenging under on-device budget constraints and biased dataset collection, especially in keyword spotting. To address this, we pr…
Keyword SpottingKnowledge DistillationRepresentation LearningSpeech Representation LearningASR-free CNN-DTW keyword spotting using multilingual bottleneck features for almost zero-resource languages
We consider multilingual bottleneck features (BNFs) for nearly zero-resource keyword spotting. This forms part of a United Nations effort using keyword spotting to support humanitarian relief programmes in parts of Afric…
Dynamic Time WarpingHumanitarianKeyword SpottingTemplate MatchingNeural Architecture Search For Keyword Spotting
Deep neural networks have recently become a popular solution to keyword spotting systems, which enable the control of smart devices via voice. In this paper, we apply neural architecture search to search for convolutiona…
Keyword SpottingNeural Architecture Search