Evolution of a Web-Scale Near Duplicate Image Detection System
Detecting near duplicate images is fundamental to the content ecosystem of photo sharing web applications. However, such a task is challenging when involving a web-scale image corpus containing billions of images. In this paper, we present an efficient system for detecting near duplicate images across 8 billion images. Our system consists of three stages: candidate generation, candidate selection, and clustering. We also demonstrate that this system can be used to greatly improve the quality of recommendations and search results across a number of real-world applications. In addition, we include the evolution of the system over the course of six years, bringing out experiences and lessons on how new systems are designed to accommodate organic content growth as well as the latest technology. Finally, we are releasing a human-labeled dataset of ~53,000 pairs of images introduced in this paper.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Benchmarking unsupervised near-duplicate image detection
Unsupervised near-duplicate detection has many practical applications ranging from social media analysis and web-scale retrieval, to digital image forensics. It entails running a threshold-limited query on a set of descr…
BenchmarkingBinary ClassificationContent-Based Image RetrievalImage Forensics+3A Review on Near Duplicate Detection of Images using Computer Vision Techniques
Nowadays, digital content is widespread and simply redistributable, either lawfully or unlawfully. For example, after images are posted on the internet, other web users can modify them and then repost their versions, the…
object-detectionObject DetectionObject RecognitionBenchmarking Pretrained Vision Embeddings for Near- and Duplicate Detection in Medical Images
Near- and duplicate image detection is a critical concern in the field of medical imaging. Medical datasets often contain similar or duplicate images from various sources, which can lead to significant performance issues…
BenchmarkingRetrievalSpecificityQuality-Aware Calibration for AI-Generated Image Detection in the Wild
Significant progress has been made in detecting synthetic images, however most existing approaches operate on a single image instance and overlook a key characteristic of real-world dissemination: as viral images circula…
Transductive Learning for Near-Duplicate Image Detection in Scanned Photo Collections
This paper presents a comparative study of near-duplicate image detection techniques in a real-world use case scenario, where a document management company is commissioned to manually annotate a collection of scanned pho…
ManagementSelf-Supervised LearningTransductive Learning