In Praise of Artifice Reloaded: Caution with subjective image quality databases
Subjective image quality databases are a major source of raw data on how the visual system works in naturalistic environments. These databases describe the sensitivity of many observers to a wide range of distortions (of different nature and with different suprathreshold intensities) seen on top of a variety of natural images. They seem like a dream for the vision scientist to check the models in realistic scenarios. However, while these natural databases are great benchmarks for models developed in some other way (e.g. by using the well-controlled artificial stimuli of traditional psychophysics), they should be carefully used when trying to fit vision models. Given the high dimensionality of the image space, it is very likely that some basic phenomenon (e.g. sensitivity to distortions in certain environments) are under-represented in the database. Therefore, a model fitted on these large-scale natural databases will not reproduce these under-represented basic phenomena that could otherwise be easily illustrated with well selected artificial stimuli. In this work we study a specific example of the above statement. A wavelet+divisive normalization layer of a sensible cascade of linear+nonlinear layers fitted to maximize the correlation with subjective opinion of observers on a large image quality database fails to reproduce basic crossmasking. Here we outline a solution for this problem using artificial stimuli. Then, we show that the resulting model is also a competitive solution for the large-scale database. In line with Rust and Movshon (2005), our report (misrepresentation of basic visual phenomena in subjectively-rated natural image databases) is an additional argument in praise of artifice.
Code (0)
등록된 구현이 없습니다.
Tasks
SensitivitySimilar Papers 제목 키워드 기반
Visual Estimation of Building Condition with Patch-level ConvNets
The condition of a building is an important factor for real estate valuation. Currently, the estimation of condition is determined by real estate appraisers which makes it subjective to a certain degree. We propose a nov…
Metrics reloaded: Recommendations for image analysis validation
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflec…
Instance SegmentationMedical Image Analysisobject-detectionObject Detection+1Seeking the Unfamiliar but Memorable: Conceptual Creativity as Meta-Learning
What does it mean to create a new concept, rather than retrieve a familiar one? Repeatedly sampling a generative model at the same prompt produces variations with similar styles and typical content. We propose that creat…
Sycophantic Praise: Evaluating Excessive Praise in Language Models
Sycophancy in language models is typically studied as excessive agreement or validation, while explicit praise and flattery have received comparatively little attention. We argue that sycophantic praise is a distinct ali…
Detecting Expressions of Blame or Praise in Text
The growth of social networking platforms has drawn a lot of attentions to the need for social computing. Social computing utilises human insights for computational tasks as well as design of systems that support social …
Attribute