Improved DeepFake Detection Using Whisper Features
With a recent influx of voice generation methods, the threat introduced by audio DeepFake (DF) is ever-increasing. Several different detection methods have been presented as a countermeasure. Many methods are based on so-called front-ends, which, by transforming the raw audio, emphasize features crucial for assessing the genuineness of the audio sample. Our contribution contains investigating the influence of the state-of-the-art Whisper automatic speech recognition model as a DF detection front-end. We compare various combinations of Whisper and well-established front-ends by training 3 detection models (LCNN, SpecRNet, and MesoNet) on a widely used ASVspoof 2021 DF dataset and later evaluating them on the DF In-The-Wild dataset. We show that using Whisper-based features improves the detection for each model and outperforms recent results on the In-The-Wild dataset by reducing Equal Error Rate by 21%.
Code (1)
Tasks
Automatic Speech RecognitionDeepFake DetectionFace Swappingspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Deepfake Detection of Singing Voices With Whisper Encodings
The deepfake generation of singing vocals is a concerning issue for artists in the music industry. In this work, we propose a singing voice deepfake detection (SVDD) system, which uses noise-variant encodings of open-AI'…
DeepFake DetectionFace SwappingDeepfake Word Detection by Next-token Prediction using Fine-tuned Whisper
Deepfake speech utterances can be forged by replacing one or more words in a bona fide utterance with semantically different words synthesized with speech-generative models. While a dedicated synthetic word detector coul…
Zero-Shot to Zero-Lies: Detecting Bengali Deepfake Audio through Transfer Learning
The rapid growth of speech synthesis and voice conversion systems has made deepfake audio a major security concern. Bengali deepfake detection remains largely unexplored. In this work, we study automatic detection of Ben…
DeepFake DetectionTransfer LearningVoice ConversionSpeech SynthesisLSTM-based Whisper Detection
This article presents a whisper speech detector in the far-field domain. The proposed system consists of a long-short term memory (LSTM) neural network trained on log-filterbank energy (LFBE) acoustic features. This mode…
BenchmarkingInvestigating Prosodic Signatures via Speech Pre-Trained Models for Audio Deepfake Source Attribution
In this work, we investigate various state-of-the-art (SOTA) speech pre-trained models (PTMs) for their capability to capture prosodic signatures of the generative sources for audio deepfake source attribution (ADSD). Th…
Audio Deepfake DetectionDeepFake DetectionFace SwappingSpeaker Recognition+2