Semi-Supervised Federated Learning for Keyword Spotting
Keyword Spotting (KWS) is a critical aspect of audio-based applications on mobile devices and virtual assistants. Recent developments in Federated Learning (FL) have significantly expanded the ability to train machine learning models by utilizing the computational and private data resources of numerous distributed devices. However, existing FL methods typically require that devices possess accurate ground-truth labels, which can be both expensive and impractical when dealing with local audio data. In this study, we first demonstrate the effectiveness of Semi-Supervised Federated Learning (SSL) and FL for KWS. We then extend our investigation to Semi-Supervised Federated Learning (SSFL) for KWS, where devices possess completely unlabeled data, while the server has access to a small amount of labeled data. We perform numerical analyses using state-of-the-art SSL, FL, and SSFL techniques to demonstrate that the performance of KWS models can be significantly improved by leveraging the abundant unlabeled heterogeneous data available on devices.
Code (1)
Tasks
Federated LearningKeyword SpottingSimilar Papers 제목 키워드 기반
Learning To Detect Keyword Parts And Whole By Smoothed Max Pooling
We propose smoothed max pooling loss and its application to keyword spotting systems. The proposed approach jointly trains an encoder (to detect keyword parts) and a decoder (to detect whole keyword) in a semi-supervised…
DecoderKeyword Spottingspeech-recognitionSpeech RecognitionProduction federated keyword spotting via distillation, filtering, and joint federated-centralized training
We trained a keyword spotting model using federated learning on real user devices and observed significant improvements when the model was deployed for inference on phones. To compensate for data domains that are missing…
Federated LearningKeyword SpottingOn-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation
Large self-supervised models are effective feature extractors, but their application is challenging under on-device budget constraints and biased dataset collection, especially in keyword spotting. To address this, we pr…
Keyword SpottingKnowledge DistillationRepresentation LearningSpeech Representation LearningTraining Keyword Spotting Models on Non-IID Data with Federated Learning
We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a centrally-trained model. To overcome the a…
Data AugmentationFederated LearningKeyword SpottingBBS-KWS:The Mandarin Keyword Spotting System Won the Video Keyword Wakeup Challenge
This paper introduces the system submitted by the Yidun NISP team to the video keyword wakeup challenge. We propose a mandarin keyword spotting system (KWS) with several novel and effective improvements, including a big …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Keyword Spottingspeech-recognition+1