Phonetic Feedback for Speech Enhancement With and Without Parallel Speech Data
While deep learning systems have gained significant ground in speech enhancement research, these systems have yet to make use of the full potential of deep learning systems to provide high-level feedback. In particular, phonetic feedback is rare in speech enhancement research even though it includes valuable top-down information. We use the technique of mimic loss to provide phonetic feedback to an off-the-shelf enhancement system, and find gains in objective intelligibility scores on CHiME-4 data. This technique takes a frozen acoustic model trained on clean speech to provide valuable feedback to the enhancement model, even in the case where no parallel speech data is available. Our work is one of the first to show intelligibility improvement for neural enhancement systems without parallel speech data, and we show phonetic feedback can improve a state-of-the-art neural enhancement system trained with parallel speech data.
Code (1)
Tasks
Speech EnhancementSimilar Papers 제목 키워드 기반
Improving Speech Enhancement Performance by Leveraging Contextual Broad Phonetic Class Information
Previous studies have confirmed that by augmenting acoustic features with the place/manner of articulatory features, the speech enhancement (SE) process can be guided to consider the broad phonetic properties of the inpu…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DenoisingMulti-Task Learning+5A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement
Speech enhancement has seen great improvement in recent years using end-to-end neural networks. However, most models are agnostic to the spoken phonetic content. Recently, several studies suggested phonetic-aware speech …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Model OptimizationSelf-Supervised Learning+2Incorporating Broad Phonetic Information for Speech Enhancement
In noisy conditions, knowing speech contents facilitates listeners to more effectively suppress background noise components and to retrieve pure speech signals. Previous studies have also confirmed the benefits of incorp…
DenoisingSpeech EnhancementImproving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement
Speech enhancement (SE) aims to improve speech quality and intelligibility, which are both related to a smooth transition in speech segments that may carry linguistic information, e.g. phones and syllables. In this study…
Speech EnhancementPAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement
Despite rapid advancement in recent years, current speech enhancement models often produce speech that differs in perceptual quality from real clean speech. We propose a learning objective that formalizes differences in …
Speech EnhancementTime SeriesTime Series Analysis