paper-with-me

홈 › Papers

Zero-Shot Temporal Localisation of Audio Deepfakes in Multi-Speaker Conversations

2026-09-09 · Soumyadeep Roy arxiv

Voice-cloning fraud increasingly relies on surgical injection: a genuine conversation in which only one or two sentences are replaced by synthetic speech. Utterance-level deepfake detectors emit a single real/fake label per clip and cannot report where the synthetic speech lies. We formalise this as Temporal Deepfake Localisation in Multi-Speaker Conversations (TDLMC), show that equal error rate and min-DCF are ill-posed once a file contains both classes, and propose temporal metrics for this regime. Our contribution is a training-free five-stage pipeline that wraps a frozen binary detector and adds segment-level output with no retraining, using a two-threshold hysteresis finitestate-machine decoder to turn noisy window scores into coherent intervals. On 180 constructed multi-speaker conversations from ASVspoof 5, the system attains temporal intersection-over-union 0.90, temporal detection rate 0.95, and MS-DCF 0.26 with a strong backbone, and its false-alarm rate on genuine speech is below 6%, falling under 2% on genuine real multi-speaker dialogue (AMI). Under an identical pipeline, a trained localiser improves temporal IoU by only about 0.04, bounding the cost of forgoing supervision. Evaluated across three frozen detectors under one decoder whose constants are selected on a held-out calibration split, and with a controlled analysis attributing the residual false-alarm rate to a backbone domain gap rather than to the decoder, this provides the first zero-shot baseline and a reusable benchmark for TDLMC.

📄 PDF Abstract BibTeX arXiv:2609.10051

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Zero-Shot to Zero-Lies: Detecting Bengali Deepfake Audio through Transfer Learning

2025-12-25 · Most. Sharmin Sultana Samu, Md. Rakibul Islam, Md. Zahid Hossain, Md. Kamrozzaman Bhuiyan 외 arxiv

The rapid growth of speech synthesis and voice conversion systems has made deepfake audio a major security concern. Bengali deepfake detection remains largely unexplored. In this work, we study automatic detection of Ben…

DeepFake DetectionTransfer LearningVoice ConversionSpeech Synthesis

Multi-label Zero-Shot Audio Classification with Temporal Attention

2024-08-31 · Duygu Dogan, Huang Xie, Toni Heittola, Tuomas Virtanen

Zero-shot learning models are capable of classifying new classes by transferring knowledge from the seen classes using auxiliary information. While most of the existing zero-shot learning methods focused on single-label …

Audio ClassificationClassificationZero-shot Audio Classificationzero-shot-classification+1

Temporal and cross-modal attention for audio-visual zero-shot learning

2022-07-20 · Otniel-Bogdan Mercea, Thomas Hummel, A. Sophia Koepke, Zeynep Akata

Audio-visual generalised zero-shot learning for video classification requires understanding the relations between the audio and visual information in order to be able to recognise samples from novel, previously unseen cl…

GZSL Video ClassificationVideo ClassificationZero-Shot Learning

Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies

2025-04-02 · Soumyya Kanti Datta, Shan Jia, Siwei Lyu

Deepfakes are AI-generated media in which the original content is digitally altered to create convincing but manipulated images, videos, or audio. Among the various types of deepfakes, lip-syncing deepfakes are one of th…

Face Swapping

Detecting Audio-Visual Deepfakes with Fine-Grained Inconsistencies

2024-08-13 · Marcella Astrid, Enjie Ghorbel, Djamila Aouada

Existing methods on audio-visual deepfake detection mainly focus on high-level features for modeling inconsistencies between audio and visual data. As a result, these approaches usually overlook finer audio-visual artifa…

DeepFake DetectionFace Swapping