DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancement

2026-03-02 · Minghui Wu, Xueling Liu, Jiahuan Fan, Haitao Tang, Yanyong Zhang, Yue Zhang arxiv

Dysarthric speech exhibits abnormal prosody and significant speaker variability, presenting persistent challenges for automatic speech recognition (ASR). While text-to-speech (TTS)-based data augmentation has shown potential, existing methods often fail to accurately model the pathological rhythm and acoustic style of dysarthric speech. To address this, we propose DARS, a dysarthria-aware rhythm-style synthesis framework based on the Matcha-TTS architecture. DARS incorporates a multi-stage rhythm predictor optimized by contrastive preferences between normal and dysarthric speech, along with a dysarthric-style conditional flow matching mechanism, jointly enhancing temporal rhythm reconstruction and pathological acoustic style simulation. Experiments on the TORGO dataset demonstrate that DARS achieves a Mean Cepstral Distortion (MCD) of 4.29, closely approximating real dysarthric speech. Adapting a Whisper-based ASR system with synthetic dysarthric speech from DARS achieves a 54.22% relative reduction in word error rate (WER) compared to state-of-the-art methods, demonstrating the framework's effectiveness in enhancing recognition performance.

📄 PDF Abstract BibTeX arXiv:2603.01369

Code (0)

등록된 구현이 없습니다.

Tasks

Speech Recognition Data Augmentation

AS-Speech: Adaptive Style For Speech Synthesis

2024-09-09 · Zhipeng Li, Xiaofen Xing, Jun Wang, Shuaiqi Chen 외

In recent years, there has been significant progress in Text-to-Speech (TTS) synthesis technology, enabling the high-quality synthesis of voices in common scenarios. In unseen situations, adaptive TTS requires a strong g…

Rhythm Speech Synthesis text-to-speech Text to Speech+1

Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens

2019-10-26 · Rafael Valle, Jason Li, Ryan Prenger, Bryan Catanzaro

Mellotron is a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data. By explicitly conditioning on rhythm and continuous pitch contours …

Rhythm Style Transfer

Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings

2022-10-04 · Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen 외

Automatic synthesis of realistic co-speech gestures is an increasingly important yet challenging task in artificial embodied agent creation. Previous systems mainly focus on generating gestures in an end-to-end manner, w…

Gesture Generation Rhythm

MimicParts: Part-aware Style Injection for Speech-Driven 3D Motion Generation

2025-10-15 · Lianlian Liu, YongKang He, Zhaojie Chu, Xiaofen Xing 외 arxiv

Generating stylized 3D human motion from speech signals presents substantial challenges, primarily due to the intricate and fine-grained relationships among speech signals, individual styles, and the corresponding body m…

Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech

2025-06-02 · Karl El Hajal, Enno Hermann, Sevada Hovsepyan, Mathew Magimai. -Doss

Automatic speech recognition (ASR) systems struggle with dysarthric speech due to high inter-speaker variability and slow speaking rates. To address this, we explore dysarthric-to-healthy speech conversion for improved A…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Rhythm speech-recognition+2