Target Speech Extraction with Conditional Diffusion Model
Diffusion model-based speech enhancement has received increased attention since it can generate very natural enhanced signals and generalizes well to unseen conditions. Diffusion models have been explored for several sub-tasks of speech enhancement, such as speech denoising, dereverberation, and source separation. In this paper, we investigate their use for target speech extraction (TSE), which consists of estimating the clean speech signal of a target speaker in a mixture of multi-talkers. TSE is realized by conditioning the extraction process on a clue identifying the target speaker. We show we can realize TSE using a conditional diffusion model conditioned on the clue. Besides, we introduce ensemble inference to reduce potential extraction errors caused by the diffusion process. In experiments on Libri2mix corpus, we show that the proposed diffusion model-based TSE combined with ensemble inference outperforms a comparable TSE system trained discriminatively.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingmodelSpeech DenoisingSpeech EnhancementSpeech ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlow
In target speaker extraction (TSE), we aim to recover target speech from a multi-talker mixture using a short enrollment utterance as reference. Recent studies on diffusion and flow-matching generators have improved targ…
Speech RecognitionSeparate in the Speech Chain: Cross-Modal Conditional Audio-Visual Target Speech Extraction
The integration of visual cues has revitalized the performance of the target speech extraction task, elevating it to the forefront of the field. Nevertheless, this multi-modal learning paradigm often encounters the chall…
Speech ExtractionDDTSE: Discriminative Diffusion Model for Target Speech Extraction
Diffusion models have gained attention in speech enhancement tasks, providing an alternative to conventional discriminative methods. However, research on target speech extraction under multi-speaker noisy conditions rema…
modelSpeech EnhancementSpeech ExtractionGuided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance
We propose Guided-TTS, a high-quality text-to-speech (TTS) model that does not require any transcript of target speaker using classifier guidance. Guided-TTS combines an unconditional diffusion probabilistic model with a…
speech-recognitionSpeech RecognitionSpeech Synthesistext-to-speech+2SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline
Target Speech Extraction (TSE) aims to isolate a target speaker's voice from a mixture of multiple speakers by leveraging speaker-specific cues, typically provided as auxiliary audio (a.k.a. cue audio). Although recent a…
Speech ExtractionSpeech Separation