Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos
In recent years, the spread of fake videos has brought great influence on individuals and even countries. It is important to provide robust and reliable results for fake videos. The results of conventional detection methods are not reliable and not robust for unseen videos. Another alternative and more effective way is to find the original video of the fake video. For example, fake videos from the Russia-Ukraine war and the Hong Kong law revision storm are refuted by finding the original video. We use an improved retrieval method to find the original video, named ViTHash. Specifically, tracing the source of fake videos requires finding the unique one, which is difficult when there are only small differences in the original videos. To solve the above problems, we designed a novel loss Hash Triplet Loss. In addition, we designed a tool called Localizator to compare the difference between the original traced video and the fake video. We have done extensive experiments on FaceForensics++, Celeb-DF and DeepFakeDetection, and we also have done additional experiments on our built three datasets: DAVIS2016-TL (video inpainting), VSTL (video splicing) and DFTL (similar videos). Experiments have shown that our performance is better than state-of-the-art methods, especially in cross-dataset mode. Experiments also demonstrated that ViTHash is effective in various forgery detection: video inpainting, video splicing and deepfakes. Our code and datasets have been released on GitHub: \url{https://github.com/lajlksdf/vtl}.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalTripletVideo InpaintingVideo RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Vision Transformer Hashing for Image Retrieval
Deep learning has shown a tremendous growth in hashing techniques for image retrieval. Recently, Transformer has emerged as a new architecture by utilizing self-attention without convolution. Transformer is also extended…
Deep HashingImage RetrievalQuantizationRetrievalTransHash: Transformer-based Hamming Hashing for Efficient Image Retrieval
Deep hamming hashing has gained growing popularity in approximate nearest neighbour search for large-scale image retrieval. Until now, the deep hashing for the image retrieval community has been dominated by convolutiona…
Deep HashingImage RetrievalRetrievalSelf-Supervised Video Hashing via Bidirectional Transformers
Most existing unsupervised video hashing methods are built on unidirectional models with less reliable training objectives, which underuse the correlations among frames and the similarity structure between videos. To…
DecoderRetrievalVideo RetrievalDual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval
Unsupervised video hashing usually optimizes binary codes by learning to reconstruct input videos. Such reconstruction constraint spends much effort on frame-level temporal context changes without focusing on video-level…
RetrievalSemantic RetrievalSemantic SimilaritySemantic Textual Similarity+1Deep Semantic Multimodal Hashing Network for Scalable Image-Text and Video-Text Retrievals
Hashing has been widely applied to multimodal retrieval on large-scale multimedia data due to its efficiency in computation and storage. In this article, we propose a novel deep semantic multimodal hashing network (DSMHN…
Cross-Modal RetrievalDeep HashingImage-text RetrievalRepresentation Learning+3