FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units
The rapid evolution of generative AI has increased the threat of realistic audio-visual deepfakes, demanding robust detection methods. Existing solutions primarily address unimodal (audio or visual) forgeries but struggle with multimodal manipulations due to inadequate handling of heterogeneous modality features and poor generalization across datasets. To this end, we propose a novel framework called FauForensics by introducing biologically invariant facial action units (FAUs), which is a quantitative descriptor of facial muscle activity linked to emotion physiology. It serves as forgery-resistant representations that reduce domain dependency while capturing subtle dynamics often disrupted in synthetic content. Besides, instead of comparing entire video clips as in prior works, our method computes fine-grained frame-wise audiovisual similarities via a dedicated fusion module augmented with learnable cross-modal queries. It dynamically aligns temporal-spatial lip-audio relationships while mitigating multi-modal feature heterogeneity issues. Experiments on FakeAVCeleb and LAV-DF show state-of-the-art (SOTA) performance and superior cross-dataset generalizability with up to an average of 4.83\% than existing methods.
Code (0)
등록된 구현이 없습니다.
Tasks
DeepFake DetectionFace SwappingSimilar Papers 제목 키워드 기반
MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection
Deepfakes are synthetic media generated using deep generative algorithms and have posed a severe societal and political threat. Apart from facial manipulation and synthetic voice, recently, a novel kind of deepfakes has …
DeepFake DetectionFace SwappingJoint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection
With the continuous advancement of deepfake technology, there has been a surge in the creation of realistic fake videos. Unfortunately, the malicious utilization of deepfake poses a significant threat to societal moralit…
Contrastive LearningDeepFake DetectionFace SwappingHuman Detection of DeepfakesContextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization
In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic …
DeepFake DetectionFace SwappingIntegrating Audio-Visual Features for Multimodal Deepfake Detection
Deepfakes are AI-generated media in which an image or video has been digitally modified. The advancements made in deepfake technology have led to privacy and security issues. Most deepfake detection techniques rely on th…
Binary ClassificationDeepFake DetectionFace SwappingLightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Framework
Deepfakes are AI-synthesized multimedia data that may be abused for spreading misinformation. Deepfake generation involves both visual and audio manipulation. To detect audio-visual deepfakes, previous studies commonly e…
audio-visual learningDeepFake DetectionFace SwappingMisinformation+1