MOPSA: Mixture of Prompt-Experts Based Speaker Adaptation for Elderly Speech Recognition
This paper proposes a novel Mixture of Prompt-Experts based Speaker Adaptation approach (MOPSA) for elderly speech recognition. It allows zero-shot, real-time adaptation to unseen speakers, and leverages domain knowledge tailored to elderly speakers. Top-K most distinctive speaker prompt clusters derived using K-means serve as experts. A router network is trained to dynamically combine clustered prompt-experts. Acoustic and language level variability among elderly speakers are modelled using separate encoder and decoder prompts for Whisper. Experiments on the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets suggest that online MOPSA adaptation outperforms the speaker-independent (SI) model by statistically significant word error rate (WER) or character error rate (CER) reductions of 0.86% and 1.47% absolute (4.21% and 5.40% relative). Real-time factor (RTF) speed-up ratios of up to 16.12 times are obtained over offline batch-mode adaptation.
Code (0)
등록된 구현이 없습니다.
Tasks
Decoderspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Exploring Speaker Diarization with Mixture of Experts
In this paper, we propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates a memory-aware multi-speaker embedding mo…
Mixture-of-Expertsspeaker-diarizationSpeaker DiarizationMAST-Pro: Dynamic Mixture-of-Experts for Adaptive Segmentation of Pan-Tumors with Knowledge-Driven Prompts
Accurate tumor segmentation is crucial for cancer diagnosis and treatment. While foundation models have advanced general-purpose segmentation, existing methods still struggle with: (1) limited incorporation of medical pr…
Mixture-of-Expertsparameter-efficient fine-tuningRepresentation LearningSegmentation+1GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
Graph Neural Networks (GNNs) have demonstrated impressive performance on task-specific benchmarks, yet their ability to generalize across diverse domains and tasks remains limited. Existing approaches often struggle with…
pMoE: Prompting Diverse Experts Together Wins More in Visual Adaptation
Parameter-efficient fine-tuning has demonstrated promising results across various visual adaptation tasks, such as classification and segmentation. Typically, prompt tuning techniques have harnessed knowledge from a sing…
parameter-efficient fine-tuningComputational EfficiencyLPT++: Efficient Training on Mixture of Long-tailed Experts
We introduce LPT++, a comprehensive framework for long-tailed classification that combines parameter-efficient fine-tuning (PEFT) with a learnable model ensemble. LPT++ enhances frozen Vision Transformers (ViTs) through …
Mixture-of-Expertsparameter-efficient fine-tuning