Benchmarking Audio Deepfake Detection Robustness in Real-world Communication Scenarios
Existing Audio Deepfake Detection (ADD) systems often struggle to generalise effectively due to the significantly degraded audio quality caused by audio codec compression and channel transmission effects in real-world communication scenarios. To address this challenge, we developed a rigorous benchmark to evaluate the performance of the ADD system under such scenarios. We introduced ADD-C, a new test dataset to evaluate the robustness of ADD systems under diverse communication conditions, including different combinations of audio codecs for compression and packet loss rates. Benchmarking three baseline ADD models on the ADD-C dataset demonstrated a significant decline in robustness under such conditions. A novel Data Augmentation (DA) strategy was proposed to improve the robustness of ADD systems. Experimental results demonstrated that the proposed approach significantly enhances the performance of ADD systems on the proposed ADD-C dataset. Our benchmark can assist future efforts towards building practical and robustly generalisable ADD systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Audio Deepfake DetectionBenchmarkingData AugmentationDeepFake DetectionFace SwappingSimilar Papers 제목 키워드 기반
Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning
Good datasets are essential for developing and benchmarking any machine learning system. Their importance is even more extreme for safety critical applications such as deepfake detection - the focus of this paper. Here w…
BenchmarkingDeepFake DetectionFace SwappingEnvSDD: Benchmarking Environmental Sound Deepfake Detection
Audio generation systems now create very realistic soundscapes that can enhance media production, but also pose potential risks. Several studies have examined deepfakes in speech or singing voice. However, environmental …
Audio Deepfake DetectionAudio GenerationBenchmarkingDeepFake Detection+1DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection
Generative AI advances rapidly, allowing the creation of very realistic manipulated video and audio. This progress presents a significant security and ethical threat, as malicious users can exploit DeepFake techniques to…
BenchmarkingDeepFake DetectionFace SwappingMisinformationJoint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection
With the continuous advancement of deepfake technology, there has been a surge in the creation of realistic fake videos. Unfortunately, the malicious utilization of deepfake poses a significant threat to societal moralit…
Contrastive LearningDeepFake DetectionFace SwappingHuman Detection of DeepfakesAUDDT: A Unified Benchmark Toolkit for Audio and Speech Deepfake Detectors
With the prevalence of artificial intelligence (AI)-generated content, such as audio deepfakes, a large body of recent work has focused on developing deepfake detection techniques. However, existing benchmarks employ a n…
DeepFake Detection