Enhancing image quality prediction with self-supervised visual masking
Full-reference image quality metrics (FR-IQMs) aim to measure the visual differences between a pair of reference and distorted images, with the goal of accurately predicting human judgments. However, existing FR-IQMs, including traditional ones like PSNR and SSIM and even perceptual ones such as HDR-VDP, LPIPS, and DISTS, still fall short in capturing the complexities and nuances of human perception. In this work, rather than devising a novel IQM model, we seek to improve upon the perceptual quality of existing FR-IQM methods. We achieve this by considering visual masking, an important characteristic of the human visual system that changes its sensitivity to distortions as a function of local image content. Specifically, for a given FR-IQM metric, we propose to predict a visual masking model that modulates reference and distorted images in a way that penalizes the visual errors based on their visibility. Since the ground truth visual masks are difficult to obtain, we demonstrate how they can be derived in a self-supervised manner solely based on mean opinion scores (MOS) collected from an FR-IQM dataset. Our approach results in enhanced FR-IQM metrics that are more in line with human prediction both visually and quantitatively.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionSSIMSimilar Papers 제목 키워드 기반
Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking
Improving vision-language models (VLMs) in the post-training stage typically relies on supervised fine-tuning or reinforcement learning, methods that necessitate costly, human-annotated data. While self-supervised techni…
Image Quality AssessmentReinforcement LearningJoint Quality Assessment and Example-Guided Image Processing by Disentangling Picture Appearance from Content
The deep learning revolution has strongly impacted low-level image processing tasks such as style/domain transfer, enhancement/restoration, and visual quality assessments. Despite often being treated separately, the afor…
Representation LearningTone MappingSelf-Paced Sample Selection for Barely-Supervised Medical Image Segmentation
The existing barely-supervised medical image segmentation (BSS) methods, adopting a registration-segmentation paradigm, aim to learn from data with very few annotations to mitigate the extreme label scarcity problem. How…
Contrastive LearningImage RegistrationImage SegmentationMedical Image Segmentation+2Self-Supervised Image Captioning with CLIP
Image captioning, a fundamental task in vision-language understanding, seeks to generate accurate natural language descriptions for provided images. Current image captioning approaches heavily rely on high-quality image-…
Image CaptioningInformativenessLanguage ModellingSelf-Supervised Learning+1SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization
Previous text-to-image diffusion models typically employ supervised fine-tuning (SFT) to enhance pre-trained base models. However, this approach primarily minimizes the loss of mean squared error (MSE) at the pixel level…
global-optimization