Improved Self-Supervised Multilingual Speech Representation Learning Combined with Auxiliary Language Information
Multilingual end-to-end models have shown great improvement over monolingual systems. With the development of pre-training methods on speech, self-supervised multilingual speech representation learning like XLSR has shown success in improving the performance of multilingual automatic speech recognition (ASR). However, similar to the supervised learning, multilingual pre-training may also suffer from language interference and further affect the application of multilingual system. In this paper, we introduce several techniques for improving self-supervised multilingual pre-training by leveraging auxiliary language information, including the language adversarial training, language embedding and language adaptive training during the pre-training stage. We conduct experiments on a multilingual ASR task consisting of 16 languages. Our experimental results demonstrate 14.3% relative gain over the standard XLSR model, and 19.8% relative gain over the no pre-training multilingual model.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Representation Learningspeech-recognitionSpeech RecognitionSpeech Representation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching
While many speakers of low-resource languages regularly code-switch between their languages and other regional languages or English, datasets of codeswitched speech are too small to train bespoke acoustic models from scr…
Language ModelingLanguage Modellingspeech-recognitionSpeech RecognitionMauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery
This paper introduces MauBERT, a multilingual extension of HuBERT that leverages articulatory features for robust cross-lingual phonetic representation learning. We continue HuBERT pre-training with supervision based on …
Self-Supervised LearningRepresentation LearningATCSpeechNet: A multilingual end-to-end speech recognition framework for air traffic control systems
In this paper, a multilingual end-to-end framework, called as ATCSpeechNet, is proposed to tackle the issue of translating communication speech into human-readable text in air traffic control (ATC) systems. In the propos…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Feature EngineeringRepresentation Learning+3Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition
State-of-the-art automatic speech recognition (ASR) systems perform well on healthy speech. However, the performance on impaired speech still remains an issue. The current study explores the usefulness of using Wav2Vec s…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionMParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting
We present MParrotTTS, a unified multilingual, multi-speaker text-to-speech (TTS) synthesis model that can produce high-quality speech. Benefiting from a modularized training paradigm exploiting self-supervised speech re…
Speech Synthesistext-to-speechText to SpeechText-To-Speech Synthesis