Analysis and Interpretation of Deep CNN Representations as Perceptual Quality Features
Pre-trained Deep Convolutional Neural Network (CNN) features have popularly been used as full-reference perceptual quality features for CNN based image quality assessment, super-resolution, image restoration and a variety of image-to-image translation problems. In this paper, to get more insight, we link basic human visual perception to characteristics of learned deep CNN representations as a novel and first attempt to interpret them. We characterize the frequency and orientation tuning of channels in trained object detection deep CNNs (e.g., VGG-16) by applying grating stimuli of different spatial frequencies and orientations as input. We observe that the behavior of CNN channels as spatial frequency and orientation selective filters can be used to link basic human visual perception models to their characteristics. Doing so, we develop a theory to get more insight into deep CNN representations as perceptual quality features. We conclude that sensitivity to spatial frequencies that have lower contrast masking thresholds in human visual perception and a definite and strong orientation selectivity are important attributes of deep CNN channels that deliver better perceptual quality features.
Code (0)
등록된 구현이 없습니다.
Tasks
Image Quality AssessmentImage RestorationImage-to-Image Translationobject-detectionObject DetectionSuper-ResolutionSimilar Papers 제목 키워드 기반
Why Are Deep Representations Good Perceptual Quality Features?
Recently, intermediate feature maps of pre-trained convolutional neural networks have shown significant perceptual quality improvements, when they are used in the loss function for training new networks. It is believed t…
Image Quality AssessmentImage ReconstructionImage RestorationImage Super-Resolution+4Learnt Deep Hyperparameter selection in Adversarial Training for compressed video enhancement with perceptual critic
Image based Deep Feature Quality Metrics (DFQMs) have been shown to better correlate with subjective perceptual scores over traditional metrics. The fundamental focus of these DFQMs is to exploit internal representations…
feature selectionVideo EnhancementPerceptual misalignment of texture representations in convolutional neural networks
Mathematical modeling of visual textures traces back to Julesz's intuition that texture perception in humans is based on local correlations between image features. An influential approach for texture analysis and generat…
Object RecognitionDExter: Learning and Controlling Performance Expression with Diffusion Models
In the pursuit of developing expressive music performance models using artificial intelligence, this paper introduces DExter, a new approach leveraging diffusion probabilistic models to render Western classical piano per…
Music Performance RenderingAgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment
Image quality assessment (IQA) is inherently complex, as it reflects both the quantification and interpretation of perceptual quality rooted in the human visual system. Conventional approaches typically rely on fixed mod…
Image Quality Assessment