Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages
Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each. However, there are thousands of spoken languages worldwide, and adapting to new languages is an important problem. In this work, we aim to understand which model adapts better to languages unseen during pre-training. We fine-tune both models on 13 unseen languages and 18 seen languages. Our results show that the number of hours seen per language and language family during pre-training is predictive of how the models compare, despite the significant differences in the pre-training methods.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Low-Resourced Speech Recognition for Iu Mien Language via Weakly-Supervised Phoneme-based Multilingual Pre-training
The mainstream automatic speech recognition (ASR) technology usually requires hundreds to thousands of hours of annotated speech data. Three approaches to low-resourced ASR are phoneme or subword based supervised pre-tra…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionXLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognition
In this paper, we propose a weakly supervised multilingual representation learning framework, called cross-lingual self-training (XLST). XLST is able to utilize a small amount of annotated data from high-resource languag…
Data AugmentationRepresentation Learningspeech-recognitionSpeech RecognitionDeploying self-supervised learning in the wild for hybrid automatic speech recognition
Self-supervised learning (SSL) methods have proven to be very successful in automatic speech recognition (ASR). These great improvements have been reported mostly based on highly curated datasets such as LibriSpeech for …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Event DetectionScheduling+3Large scale weakly and semi-supervised learning for low-resource video ASR
Many semi- and weakly-supervised approaches have been investigated for overcoming the labeling cost of building high quality speech recognition systems. On the challenging task of transcribing social media videos in low-…
Decoderspeech-recognitionSpeech RecognitionWeakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Driving
Understanding motion in dynamic environments is critical for autonomous driving, thereby motivating research on class-agnostic motion prediction. In this work, we investigate weakly and self-supervised class-agnostic mot…
Self-Supervised LearningAutonomous DrivingScene ParsingPoint Clouds