Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech
Automatic speech recognition (ASR) systems struggle with dysarthric speech due to high inter-speaker variability and slow speaking rates. To address this, we explore dysarthric-to-healthy speech conversion for improved ASR performance. Our approach extends the Rhythm and Voice (RnV) conversion framework by introducing a syllable-based rhythm modeling method suited for dysarthric speech. We assess its impact on ASR by training LF-MMI models and fine-tuning Whisper on converted speech. Experiments on the Torgo corpus reveal that LF-MMI achieves significant word error rate reductions, especially for more severe cases of dysarthria, while fine-tuning Whisper on converted data has minimal effect on its performance. These results highlight the potential of unsupervised rhythm and voice conversion for dysarthric ASR. Code available at: https://github.com/idiap/RnV
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Rhythmspeech-recognitionSpeech RecognitionVoice ConversionSimilar Papers 제목 키워드 기반
Unsupervised Rhythm and Voice Conversion of Dysarthric to Healthy Speech for ASR
Automatic speech recognition (ASR) systems are well known to perform poorly on dysarthric speech. Previous works have addressed this by speaking rate modification to reduce the mismatch with typical speech. Unfortunately…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Rhythmspeech-recognition+2Rhythm Modeling for Voice Conversion
Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in the perception of speaker identity. To …
RhythmVoice ConversionThe Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition
In this paper, we investigate several existing and a new state-of-the-art generative adversarial network-based (GAN) voice conversion method for enhancing dysarthric speech for improved dysarthric speech recognition. We …
Generative Adversarial NetworkPhoneme Recognitionspeech-recognitionSpeech Recognition+1Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages
Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conversion model on English dysarthric speec…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition+1Towards Identity Preserving Normal to Dysarthric Voice Conversion
We present a voice conversion framework that converts normal speech into dysarthric speech while preserving the speaker identity. Such a framework is essential for (1) clinical decision making processes and alleviation o…
Data AugmentationDecision Makingspeech-recognitionSpeech Recognition+1