Now It Sounds Like You: Learning Personalized Vocabulary On Device
In recent years, Federated Learning (FL) has shown significant advancements in its ability to perform various natural language processing (NLP) tasks. This work focuses on applying personalized FL for on-device language modeling. Due to limitations of memory and latency, these models cannot support the complexity of sub-word tokenization or beam search decoding, resulting in the decision to deploy a closed-vocabulary language model. However, closed-vocabulary models are unable to handle out-of-vocabulary (OOV) words belonging to specific users. To address this issue, We propose a novel technique called "OOV expansion" that improves OOV coverage and increases model accuracy while minimizing the impact on memory and latency. This method introduces a personalized "OOV adapter" that effectively transfers knowledge from a central model and learns word embedding for personalized vocabulary. OOV expansion significantly outperforms standard FL personalization methods on a set of common FL benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
SmartPhone: Exploring Keyword Mnemonic with Auto-generated Verbal and Visual Cues
In second language vocabulary learning, existing works have primarily focused on either the learning interface or scheduling personalized retrieval practices to maximize memory retention. However, the learning content, i…
RetrievalSchedulingProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
Recent advances have enabled automatic sound recognition systems for deaf and hard of hearing (DHH) users on mobile devices. However, these tools use pre-trained, generic sound recognition models, which do not meet the d…
Respiratory Inhaler Sound Event Classification Using Self-Supervised Learning
Asthma is a chronic respiratory condition that affects millions of people worldwide. While this condition can be managed by administering controller medications through handheld inhalers, clinical studies have shown low …
ClassificationSelf-Supervised LearningSound ClassificationMixture-of-Experts Framework for Field-of-View Enhanced Signal-Dependent Binauralization of Moving Talkers
We propose a novel mixture of experts framework for field-of-view enhancement in binaural signal matching. Our approach enables dynamic spatial audio rendering that adapts to continuous talker motion, allowing users to e…
Personalized Speech recognition on mobile devices
We describe a large vocabulary speech recognition system that is accurate, has low latency, and yet has a small enough memory and computational footprint to run faster than real-time on a Nexus 5 Android smartphone. We e…
DecoderLanguage ModelingLanguage Modellingspeech-recognition+1