Introducing Explicit Gaze Constraints to Face Swapping
Face swapping combines one face's identity with another face's non-appearance attributes (expression, head pose, lighting) to generate a synthetic face. This technology is rapidly improving, but falls flat when reconstructing some attributes, particularly gaze. Image-based loss metrics that consider the full face do not effectively capture the perceptually important, yet spatially small, eye regions. Improving gaze in face swaps can improve naturalness and realism, benefiting applications in entertainment, human computer interaction, and more. Improved gaze will also directly improve Deepfake detection efforts, serving as ideal training data for classifiers that rely on gaze for classification. We propose a novel loss function that leverages gaze prediction to inform the face swap model during training and compare against existing methods. We find all methods to significantly benefit gaze in resulting face swaps.
Code (0)
등록된 구현이 없습니다.
Tasks
DeepFake DetectionFace SwappingGaze PredictionSimilar Papers 제목 키워드 기반
High-resolution Face Swapping via Latent Semantics Disentanglement
We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their q…
DisentanglementFace SwappingHallucinationVocal Bursts Intensity PredictionTowards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms
Advances in face swapping have enabled the automatic generation of highly realistic faces. Yet face swaps are perceived differently than when looking at real faces, with key differences in viewer behavior surrounding the…
DeepFake DetectionFace SwappingGaze EstimationCA-IDD: Cross-Attention Guided Identity-Conditional Diffusion for Identity-Consistent Face Swapping
Face swapping aims to optimize realistic facial image generation by leveraging the identity of a source face onto a target face while preserving pose, expression, and context. However, existing methods, especially GAN-ba…
Image GenerationFace SwappingUnsupervised Gaze Representation Learning from Multi-view Face Images
Annotating gaze is an expensive and time-consuming endeavor requiring costly eye-trackers or complex geometric calibration procedures. Although some eye-based unsupervised gaze representation learning methods have be…
Gaze EstimationRepresentation LearningSimSwap: An Efficient Framework For High Fidelity Face Swapping
We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity…
Face SwappingVocal Bursts Intensity Prediction