Text Modality Oriented Image Feature Extraction for Detecting Diffusion-based DeepFake
The widespread use of diffusion methods enables the creation of highly realistic images on demand, thereby posing significant risks to the integrity and safety of online information and highlighting the necessity of DeepFake detection. Our analysis of features extracted by traditional image encoders reveals that both low-level and high-level features offer distinct advantages in identifying DeepFake images produced by various diffusion methods. Inspired by this finding, we aim to develop an effective representation that captures both low-level and high-level features to detect diffusion-based DeepFakes. To address the problem, we propose a text modality-oriented feature extraction method, termed TOFE. Specifically, for a given target image, the representation we discovered is a corresponding text embedding that can guide the generation of the target image with a specific text-to-image model. Experiments conducted across ten diffusion types demonstrate the efficacy of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
DeepFake DetectionFace SwappingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
EI-Nexus: Towards Unmediated and Flexible Inter-Modality Local Feature Extraction and Matching for Event-Image Data
Event cameras, with high temporal resolution and high dynamic range, have limited research on the inter-modality local feature extraction and matching of event-image data. We propose EI-Nexus, an unmediated and flexible …
Pose EstimationListen to Both Sides and be Enlightened! -- Hierarchical Modality Fusion Network for Entity and Relation Extraction
Multimodal named entity recognition and relation extraction (MNER and MRE) is a fundamental and crucial branch in multimodal learning. However, existing approaches for MNER and MRE mainly suffer from 1) error sensitivi…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Relation+1MO R-CNN: Multispectral Oriented R-CNN for Object Detection in Remote Sensing Image
Oriented object detection for multi-spectral imagery faces significant challenges due to differences both within and between modalities. Although existing methods have improved detection accuracy through complex network …
Object DetectionSpatial Dual-Modality Graph Reasoning for Key Information Extraction
Key information extraction from document images is of paramount importance in office automation. Conventional template matching based approaches fail to generalize well to document images of unseen templates, and are not…
Key Information ExtractionTemplate MatchingLow-Level Features for Image Retrieval Based on Extraction of Directional Binary Patterns and Its Oriented Gradients Histogram
In this paper, we present a novel approach for image retrieval based on extraction of low level features using techniques such as Directional Binary Code, Haar Wavelet transform and Histogram of Oriented Gradients. The D…
Content-Based Image RetrievalImage RetrievalRetrieval