Visual Analysis Motivated Rate-Distortion Model for Image Coding
Optimized for pixel fidelity metrics, images compressed by existing image codec are facing systematic challenges when used for visual analysis tasks, especially under low-bitrate coding. This paper proposes a visual analysis-motivated rate-distortion model for Versatile Video Coding (VVC) intra compression. The proposed model has two major contributions, a novel rate allocation strategy and a new distortion measurement model. We first propose the region of interest for machine (ROIM) to evaluate the degree of importance for each coding tree unit (CTU) in visual analysis. Then, a novel CTU-level bit allocation model is proposed based on ROIM and the local texture characteristics of each CTU. After an in-depth analysis of multiple distortion models, a visual analysis friendly distortion criteria is subsequently proposed by extracting deep feature of each coding unit (CU). To alleviate the problem of lacking spatial context information when calculating the distortion of each CU, we finally propose a multi-scale feature distortion (MSFD) metric using different neighboring pixels by weighting the extracted deep features in each scale. Extensive experimental results show that the proposed scheme could achieve up to 28.17\% bitrate saving under the same analysis performance among several typical visual analysis tasks such as image classification, object detection, and semantic segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage Classificationobject-detectionObject DetectionSemantic SegmentationSimilar Papers 제목 키워드 기반
Regression-free Blind Image Quality Assessment with Content-Distortion Consistency
The optimization objective of regression-based blind image quality assessment (IQA) models is to minimize the mean prediction error across the training dataset, which can lead to biased parameter estimation due to potent…
Image Quality AssessmentNo-Reference Image Quality Assessmentparameter estimationregression+2ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images
Recent advances in Multimodal Large Language Models (MLLMs) have introduced a paradigm shift for Image Quality Assessment (IQA) from unexplainable image quality scoring to explainable IQA, demonstrating practical applica…
Image Quality AssessmentImage RestorationA Rate-Distortion-Classification Approach for Lossy Image Compression
In lossy image compression, the objective is to achieve minimal signal distortion while compressing images to a specified bit rate. The increasing demand for visual analysis applications, particularly in classification t…
ClassificationImage CompressionRestoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
In recent years, deep learning-based methods have been successfully applied to the image distortion restoration tasks. However, scenarios that assume a single distortion only may not be suitable for many real-world appli…
Mixture-of-ExpertsMulti-Task LearningVLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression
Evaluations of image compression performance which include human preferences have generally found that naive distortion functions such as MSE are insufficiently aligned to human perception. In order to align compression …
Image CompressionVisual Reasoning