Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking
Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human audio datasets. We address this by investigating synthetic audio data for audio-based DST. To this end, we develop cascading and end-to-end models, train them with our synthetic audio dataset, and test them on actual human speech data. To facilitate evaluation tailored to audio modalities, we introduce a novel PhonemeF1 to capture pronunciation similarity. Experimental results showed that models trained solely on synthetic datasets can generalize their performance to human voice data. By eliminating the dependency on human speech data collection, these insights pave the way for significant practical advancements in audio-based DST. Data and code are available at https://github.com/JihyunLee1/E2E-DST.
Code (1)
Tasks
Dialogue State TrackingTask-Oriented Dialogue SystemsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Synthetic Speech Source Tracing using Metric Learning
This paper addresses source tracing in synthetic speech-identifying generative systems behind manipulated audio via speaker recognition-inspired pipelines. While prior work focuses on spoofing detection, source tracing l…
Metric LearningSelf-Supervised LearningSpeaker RecognitionInvestigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection
While Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have shown strong generalisation in detecting image and video deepfakes, their use for audio deepfake detection remains largely unexplored.…
Audio Deepfake DetectionASPED: An Audio Dataset for Detecting Pedestrians
We introduce the new audio analysis task of pedestrian detection and present a new large-scale dataset for this task. While the preliminary results prove the viability of using audio approaches for pedestrian detection, …
Pedestrian DetectionIteraTTA: An interface for exploring both text prompts and audio priors in generating music with text-to-audio models
Recent text-to-audio generation techniques have the potential to allow novice users to freely generate music audio. Even if they do not have musical knowledge, such as about chord progressions and instruments, users can …
Audio GenerationMusic GenerationFlexible framework for audio reconstruction
The paper presents a unified, flexible framework for the tasks of audio inpainting, declipping, and dequantization. The concept is further extended to cover analogous degradation models in a transformed domain, e.g. quan…
Audio inpaintingQuantization