Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines
Deepfakes pose a serious threat to digital well-being by fueling misinformation. As deepfakes get harder to recognize with the naked eye, human users become increasingly reliant on deepfake detection models to decide if a video is real or fake. Currently, models yield a prediction for a video's authenticity, but do not integrate a method for alerting a human user. We introduce a framework for amplifying artifacts in deepfake videos to make them more detectable by people. We propose a novel, semi-supervised Artifact Attention module, which is trained on human responses to create attention maps that highlight video artifacts. These maps make two contributions. First, they improve the performance of our deepfake detection classifier. Second, they allow us to generate novel "Deepfake Caricatures": transformations of the deepfake that exacerbate artifacts to improve human detection. In a user study, we demonstrate that Caricatures greatly increase human detection, across video presentation times and user engagement levels. Overall, we demonstrate the success of a human-centered approach to designing deepfake mitigation methods.
Code (0)
등록된 구현이 없습니다.
Tasks
DeepFake DetectionFace SwappingHuman DetectionMisinformationSimilar Papers 제목 키워드 기반
Pay Less Attention to Deceptive Artifacts: Robust Detection of Compressed Deepfakes on Online Social Networks
With the rapid advancement of deep learning, particularly through generative adversarial networks (GANs) and diffusion models (DMs), AI-generated images, or ``deepfakes", have become nearly indistinguishable from real on…
DeepFake DetectionFace SwappingFrom Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques. In this paper, we ask: How can we build a universal detection framework that is effective …
Face SwappingSpecificitySuper-ResolutionIn Anticipation of Perfect Deepfake: Identity-anchored Artifact-agnostic Detection under Rebalanced Deepfake Detection Protocol
As deep generative models advance, we anticipate deepfakes achieving "perfection"-generating no discernible artifacts or noise. However, current deepfake detectors, intentionally or inadvertently, rely on such artifacts …
DeepFake DetectionFace SwappingBeyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection
We introduce a deepfake video detection approach that exploits pixel-wise temporal inconsistencies, which traditional spatial frequency-based detectors often overlook. Traditional detectors represent temporal information…
Face SwappingExposing DeepFake Videos By Detecting Face Warping Artifacts
In this work, we describe a new deep learning based method that can effectively distinguish AI-generated fake videos (referred to as {\em DeepFake} videos hereafter) from real videos. Our method is based on the observati…
Face Swapping