End-to-end Anchored Speech Recognition
Voice-controlled house-hold devices, like Amazon Echo or Google Home, face the problem of performing speech recognition of device-directed speech in the presence of interfering background speech, i.e., background noise and interfering speech from another person or media device in proximity need to be ignored. We propose two end-to-end models to tackle this problem with information extracted from the "anchored segment". The anchored segment refers to the wake-up word part of an audio stream, which contains valuable speaker information that can be used to suppress interfering speech and background noise. The first method is called "Multi-source Attention" where the attention mechanism takes both the speaker information and decoder state into consideration. The second method directly learns a frame-level mask on top of the encoder output. We also explore a multi-task learning setup where we use the ground truth of the mask to guide the learner. Given that audio data with interfering speech is rare in our training data set, we also propose a way to synthesize "noisy" speech from "clean" speech to mitigate the mismatch between training and test data. Our proposed methods show up to 15% relative reduction in WER for Amazon Alexa live data with interfering background speech without significantly degrading on clean speech.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMulti-Task Learningspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Anchored Speech Recognition with Neural Transducers
Neural transducers have achieved human level performance on standard speech recognition benchmarks. However, their performance significantly degrades in the presence of cross-talk, especially when the primary speaker has…
speech-recognitionSpeech RecognitionSoft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
Joint Embedding Predictive Architectures (JEPA) offer a promising approach to self-supervised speech representation learning, but suffer from representation collapse without explicit grounding. We propose GMM-Anchored JE…
Representation LearningEmotion RecognitionSlot FillingCRUSH: Contextually Regularized and User anchored Self-supervised Hate speech Detection
The last decade has witnessed a surge in the interaction of people through social networking platforms. While there are several positive aspects of these social platforms, the proliferation has led them to become the bre…
Hate Speech DetectionAcoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition
Instruction-following audio language models (ALMs) can be augmented with explicit acoustic cues, yet it remains unclear whether such cues are used in a grounded way when the raw audio is already available. We study this …
Speech Emotion RecognitionVAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
Omni-modal large language models (OLLMs) offer a promising end-to-end solution for slide-enhanced speech recognition due to their inherent multimodal capabilities. However, we found a fundamental issue faced by OLLMs: \t…
Reinforcement LearningSpeech Recognition