paper-with-me

홈 › Papers

Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes

2024-12-17 · Kuiyuan Zhang, Zhongyun Hua, Rushi Lan, Yushu Zhang, Yifang Guo

Recent advancements in text-to-speech and speech conversion technologies have enabled the creation of highly convincing synthetic speech. While these innovations offer numerous practical benefits, they also cause significant security challenges when maliciously misused. Therefore, there is an urgent need to detect these synthetic speech signals. Phoneme features provide a powerful speech representation for deepfake detection. However, previous phoneme-based detection approaches typically focused on specific phonemes, overlooking temporal inconsistencies across the entire phoneme sequence. In this paper, we develop a new mechanism for detecting speech deepfakes by identifying the inconsistencies of phoneme-level speech features. We design an adaptive phoneme pooling technique that extracts sample-specific phoneme-level features from frame-level speech data. By applying this technique to features extracted by pre-trained audio models on previously unseen deepfake datasets, we demonstrate that deepfake samples often exhibit phoneme-level inconsistencies when compared to genuine speech. To further enhance detection accuracy, we propose a deepfake detector that uses a graph attention network to model the temporal dependencies of phoneme-level features. Additionally, we introduce a random phoneme substitution augmentation technique to increase feature diversity during training. Extensive experiments on four benchmark datasets demonstrate the superior performance of our method over existing state-of-the-art detection methods.

📄 PDF Abstract BibTeX arXiv:2412.12619

Code (0)

등록된 구현이 없습니다.

Tasks

DeepFake DetectionFace SwappingGraph Attentiontext-to-speechText to Speech

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

Generalized Multilingual Text-to-Speech Generation with Language-Aware Style Adaptation

2025-04-11 · Haowei Lou, Hye-Young Paik, Sheng Li, Wen Hu 외

Text-to-Speech (TTS) models can generate natural, human-like speech across multiple languages by transforming phonemes into waveforms. However, multilingual TTS remains challenging due to discrepancies in phoneme vocabul…

text-to-speechText to Speech

Phoneme-Level Deepfake Detection Across Emotional Conditions Using Self-Supervised Embeddings

2026-05-04 · Vamshi Nallaguntla, Shruti Kshirsagar, Anderson R. Avila arxiv

Recent advances in emotional voice conversion (EVC) have enabled the generation of expressive synthetic speech, raising new concerns in audio deepfake detection. Existing approaches treat speech as a homogeneous signal a…

Audio Deepfake DetectionVoice Conversion

Metric Learning for Phoneme Perception

2018-09-20 · Yair Lakretz, Gal Chechik, Evan-Gary Cohen, Alessandro Treves 외

Metric functions for phoneme perception capture the similarity structure among phonemes in a given language and therefore play a central role in phonology and psycho-linguistics. Various phenomena depend on phoneme simil…

Metric Learning

PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis

2025-10-16 · Soumyya Kanti Datta, Tanvi Ranga, Chengzhe Sun, Siwei Lyu arxiv

The rise of manipulated media has made deepfakes a particularly insidious threat, involving various generative manipulations such as lip-sync modifications, face-swaps, and avatar-driven facial synthesis. Conventional de…

DeepFake Detection

Stability and Fluctuations in a Simple Model of Phonetic Category Change

2017-04-20 · Benjamin Goodman, Paul Tupper

In spoken languages, speakers divide up the space of phonetic possibilities into different regions, corresponding to different phonemes. We consider a simple exemplar model of how this division of phonetic space varies o…