Language-Universal Speech Attributes Modeling for Zero-Shot Multilingual Spoken Keyword Recognition
We propose a novel language-universal approach to end-to-end automatic spoken keyword recognition (SKR) leveraging upon (i) a self-supervised pre-trained model, and (ii) a set of universal speech attributes (manner and place of articulation). Specifically, Wav2Vec2.0 is used to generate robust speech representations, followed by a linear output layer to produce attribute sequences. A non-trainable pronunciation model then maps sequences of attributes into spoken keywords in a multilingual setting. Experiments on the Multilingual Spoken Words Corpus show comparable performances to character- and phoneme-based SKR in seen languages. The inclusion of domain adversarial training (DAT) improves the proposed framework, outperforming both character- and phoneme-based SKR approaches with 13.73% and 17.22% relative word error rate (WER) reduction in seen languages, and achieves 32.14% and 19.92% WER reduction for unseen languages in zero-shot settings.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network
The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribu…
AttributeSpeaker VerificationZero-shot Learning for Speech Recognition with Universal Phonetic Model
There are more than 7,000 languages in the world, but due to the lack of training sets, only a small number of them have speech recognition systems. Multilingual speech recognition provides a solution if at least some au…
speech-recognitionSpeech RecognitionZero-Shot LearningCUPE: Contextless Universal Phoneme Encoder for Language-Agnostic Speech Processing
Universal phoneme recognition typically requires analyzing long speech segments and language-specific patterns. Many speech processing tasks require pure phoneme representations free from contextual influence, which moti…
Automatic Speech Recognition and Topic Identification for Almost-Zero-Resource Languages
Automatic speech recognition (ASR) systems often need to be developed for extremely low-resource languages to serve end-uses such as audio content categorization and search. While universal phone recognition is natural t…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Humanitarianspeech-recognition+1Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations
Current large speech language models are mainly based on semantic tokens from discretization of self-supervised learned representations and acoustic tokens from a neural codec, following a semantic-modeling and acoustic-…