Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition
While pre-trained automatic speech recognition (ASR) systems demonstrate impressive performance on matched domains, their performance often degrades when confronted with channel mismatch stemming from unseen recording environments and conditions. To mitigate this issue, we propose a novel channel-aware data simulation method for robust ASR training. Our method harnesses the synergistic power of channel-extractive techniques and generative adversarial networks (GANs). We first train a channel encoder capable of extracting embeddings from arbitrary audio. On top of this, channel embeddings are extracted using a minimal amount of target-domain data and used to guide a GAN-based speech synthesizer. This synthesizer generates speech that faithfully preserves the phonetic content of the input while mimicking the channel characteristics of the target domain. We evaluate our method on the challenging Hakka Across Taiwan (HAT) and Taiwanese Across Taiwan (TAT) corpora, achieving relative character error rate (CER) reductions of 20.02% and 9.64%, respectively, compared to the baselines. These results highlight the efficacy of our channel-aware data simulation method for bridging the gap between source- and target-domain acoustics.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Generative Adversarial NetworkRobust Speech Recognitionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Learning Semantic-aware Normalization for Generative Adversarial Networks
The recent advances in image generation have been achieved by style-based image generators. Such approaches learn to disentangle latent factors in different image scales and encode latent factors as “style” to control im…
Image GenerationImage InpaintingUnconditional Image GenerationDTGAN: Dual Attention Generative Adversarial Networks for Text-to-Image Generation
Most existing text-to-image generation methods adopt a multi-stage modular architecture which has three significant problems: 1) Training multiple networks increases the run time and affects the convergence and stability…
Generative Adversarial NetworkImage GenerationSentenceText to Image Generation+1Unpaired Image Enhancement with Quality-Attention Generative Adversarial Network
In this work, we aim to learn an unpaired image enhancement model, which can enrich low-quality images with the characteristics of high-quality images provided by users. We propose a quality attention generative adversar…
Generative Adversarial NetworkImage EnhancementMultisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification
Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions and reduce hand-crafted cost in the field of remote sensing. However, existing approa…
DiversityDomain Generalizationimage-classificationImage Classification+3Self Sparse Generative Adversarial Networks
Generative Adversarial Networks (GANs) are an unsupervised generative model that learns data distribution through adversarial training. However, recent experiments indicated that GANs are difficult to train due to the re…
Generative Adversarial NetworkImage Generation