Understanding Subjectivity through the Lens of Motivational Context in Model-Generated Image Satisfaction
Image generation models are poised to become ubiquitous in a range of applications. These models are often fine-tuned and evaluated using human quality judgments that assume a universal standard, failing to consider the subjectivity of such tasks. To investigate how to quantify subjectivity, and the scale of its impact, we measure how assessments differ among human annotators across different use cases. Simulating the effects of ordinarily latent elements of annotators subjectivity, we contrive a set of motivations (t-shirt graphics, presentation visuals, and phone background images) to contextualize a set of crowdsourcing tasks. Our results show that human evaluations of images vary within individual contexts and across combinations of contexts. Three key factors affecting this subjectivity are image appearance, image alignment with text, and representation of objects mentioned in the text. Our study highlights the importance of taking individual users and contexts into account, both when building and evaluating generative models
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Facts-and-Feelings: Capturing both Objectivity and Subjectivity in Table-to-Text Generation
Table-to-text generation, a long-standing challenge in natural language generation, has remained unexplored through the lens of subjectivity. Subjectivity here encompasses the comprehension of information derived from th…
Table-to-Text GenerationText GenerationArchiLense: A Framework for Quantitative Analysis of Architectural Styles Based on Vision Large Language Models
Architectural cultures across regions are characterized by stylistic diversity, shaped by historical, social, and technological contexts in addition to geograph-ical conditions. Understanding architectural styles require…
DescriptiveSimulating Human-like Daily Activities with Desire-driven Autonomy
Existing task-oriented AI agents often depend on explicit instructions or external rewards, limiting their ability to be driven by intrinsic motivations like humans. In this paper, we present a desire-driven autonomy fra…
Language ModelingLanguage ModellingLarge Language ModelWhen Should an AI Act? A Human-Centered Model of Scene, Context, and Behavior for Agentic AI Design
Agentic AI increasingly intervenes proactively by inferring users' situations from contextual data yet often fails for lack of principled judgment about when, why, and whether to act. We address this gap by proposing a c…
Towards Detecting Need for Empathetic Response in Motivational Interviewing
Empathetic response from the therapist is key to the success of clinical psychotherapy, especially motivational interviewing. Previous work on computational modelling of empathy in motivational interviewing has focused o…
Position