Anomaly Detection and Localization for Speech Deepfakes via Feature Pyramid Matching
The rise of AI-driven generative models has enabled the creation of highly realistic speech deepfakes - synthetic audio signals that can imitate target speakers' voices - raising critical security concerns. Existing methods for detecting speech deepfakes primarily rely on supervised learning, which suffers from two critical limitations: limited generalization to unseen synthesis techniques and a lack of explainability. In this paper, we address these issues by introducing a novel interpretable one-class detection framework, which reframes speech deepfake detection as an anomaly detection task. Our model is trained exclusively on real speech to characterize its distribution, enabling the classification of out-of-distribution samples as synthetically generated. Additionally, our framework produces interpretable anomaly maps during inference, highlighting anomalous regions across both time and frequency domains. This is done through a Student-Teacher Feature Pyramid Matching system, enhanced with Discrepancy Scaling to improve generalization capabilities across unseen data distributions. Extensive evaluations demonstrate the superior performance of our approach compared to the considered baselines, validating the effectiveness of framing speech deepfake detection as an anomaly detection problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionDeepFake DetectionFace SwappingSimilar Papers 제목 키워드 기반
DiffFake: Exposing Deepfakes using Differential Anomaly Detection
Traditional deepfake detectors have dealt with the detection problem as a binary classification task. This approach can achieve satisfactory results in cases where samples of a given deepfake generation technique have be…
Anomaly DetectionBinary ClassificationFace SwappingDiMoDif: Discourse Modality-information Differentiation for Audio-visual Deepfake Detection and Localization
Deepfake technology has rapidly advanced, posing significant threats to information integrity and societal trust. While significant progress has been made in detecting deepfakes, the simultaneous manipulation of audio an…
DeepFake DetectionFace Swappingspeech-recognitionSpeech Recognition+1PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing
Neural speech editing enables seamless partial edits to speech utterances, allowing modifications to selected content while preserving the rest of the audio unchanged. This useful technique, however, also poses new risks…
Face SwappingAlethia: A Foundational Encoder for Voice Deepfakes
Existing voice deepfake detection and localization models rely heavily on representations extracted from speech foundation models (SFMs). However, downstream finetuning has now reached a state of diminishing returns. In …
Zero-shot GeneralizationDeepFake DetectionJoint Audio-Visual Deepfake Detection
Deepfakes ("deep learning" + "fake") are synthetically-generated videos from AI algorithms. While they could be entertaining, they could also be misused for falsifying speeches and spreading misinformation. The proce…
DeepFake DetectionFace SwappingMisinformationtext-to-speech+2