Novel Hardware Implementation of Deduplicating Visually Identical JPEG Image Chunks
The exponential growth of data, particularly JPEG images spurred by mobile and social media applications, poses significant storage challenges for data centers. Traditional deduplication methods such as FSP or CDC rely on binary data, which can differ even when two images are visually identical. PXDedup, a chunk-based image deduplication strategy, effectively addresses this issue by recognizing and eliminating visual redundancies. This approach offers substantial improvements in JPEG image deduplication. However, the software implementation of this method on standard CPUs suffers from low throughput due to the CPUs’ limited processing capabilities. This study introduces an optimized deduplication method that leverages PXDedup to target visual redundancies in JPEG images. Our research focuses on hardware acceleration for image dimensions of 512 × 512 pixels. By implementing this approach on the Digilent Genesys 2 board with Xilinx Kintex-7 FPGA, we achieve a performance of 25 MB/s, tenfold faster than single-core CPU implementation and threefold quicker than quad-core CPU setup. Additionally, our analysis indicates minimal disparity in the results across all three datasets in software and hardware implementations. This evaluation underscores the effectiveness of hardware acceleration in enhancing deduplication throughput while ensuring accuracy and reliability.
Code (0)
등록된 구현이 없습니다.
Tasks
CPUSimilar Papers 제목 키워드 기반
An FPGA Implementation of Displacement Vector Search for Intra Pattern Copy in JPEG XS
Recently, progress has been made on the Intra Pattern Copy (IPC) tool for JPEG XS, an image compression standard designed for low-latency and low-complexity coding. IPC performs wavelet-domain intra compensation predicti…
Image CompressionOverview of Variable Rate Coding in JPEG AI
Empirical evidence has demonstrated that learning-based image compression can outperform classical compression frameworks. This has led to the ongoing standardization of learned-based image codecs, namely Joint Photograp…
Image CompressionCommittee Draft of JPEG XL Image Coding System
JPEG XL is a practical approach focused on scalable web distribution and efficient compression of high-quality images. It provides various benefits compared to existing image formats: 60% size reduction at equivalent sub…
Guetzli: Perceptually Guided JPEG Encoder
Guetzli is a new JPEG encoder that aims to produce visually indistinguishable images at a lower bit-rate than other common JPEG encoders. It optimizes both the JPEG global quantization tables and the DCT coefficient valu…
Perceptual DistanceQuantizationEvaluating the Practicality of Learned Image Compression
Learned image compression has achieved extraordinary rate-distortion performance in PSNR and MS-SSIM compared to traditional methods. However, it suffers from intensive computation, which is intolerable for real-world ap…
CPUGPUImage CompressionMS-SSIM+3