Generative Adversarial Training Data Adaptation for Very Low-resource Automatic Speech Recognition
It is important to transcribe and archive speech data of endangered languages for preserving heritages of verbal culture and automatic speech recognition (ASR) is a powerful tool to facilitate this process. However, since endangered languages do not generally have large corpora with many speakers, the performance of ASR models trained on them are considerably poor in general. Nevertheless, we are often left with a lot of recordings of spontaneous speech data that have to be transcribed. In this work, for mitigating this speaker sparsity problem, we propose to convert the whole training speech data and make it sound like the test speaker in order to develop a highly accurate ASR system for this speaker. For this purpose, we utilize a CycleGAN-based non-parallel voice conversion technology to forge a labeled training data that is close to the test speaker's speech. We evaluated this speaker adaptation approach on two low-resource corpora, namely, Ainu and Mboshi. We obtained 35-60% relative improvement in phone error rate on the Ainu corpus, and 40% relative improvement was attained on the Mboshi corpus. This approach outperformed two conventional methods namely unsupervised adaptation and multilingual training with these two corpora.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Cultural Vocal Bursts Intensity Predictionspeech-recognitionSpeech RecognitionVoice ConversionSimilar Papers 제목 키워드 기반
A Generative Framework for Zero-Shot Learning with Adversarial Domain Adaptation
We present a domain adaptation based generative framework for zero-shot learning. Our framework addresses the problem of domain shift between the seen and unseen class distributions in zero-shot learning and minimizes th…
AttributeDomain AdaptationZero-Shot LearningSemantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation
In spite of the compelling achievements that deep neural networks (DNNs) have made in medical image computing, these deep models often suffer from degraded performance when being applied to new test datasets with domain …
Domain AdaptationSegmentationTransfer LearningUnsupervised Domain AdaptationZero-shot Generative Model Adaptation via Image-specific Prompt Learning
Recently, CLIP-guided image synthesis has shown appealing performance on adapting a pre-trained source-domain generator to an unseen target domain. It does not require any target-domain samples but only the textual domai…
DiversityImage GenerationPrompt LearningPreserving Semantic Consistency in Unsupervised Domain Adaptation Using Generative Adversarial Networks
Unsupervised domain adaptation seeks to mitigate the distribution discrepancy between source and target domains, given labeled samples of the source domain and unlabeled samples of the target domain. Generative adversari…
Domain AdaptationGenerative Adversarial NetworkUnsupervised Domain AdaptationMAGAN: Margin Adaptation for Generative Adversarial Networks
We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the …
Image Generation