paper-with-me

Papers

Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbols

2025-11-28 · Sebastian Padó, Kerstin Thomas arxiv

Emotions are a fundamental aspect of artistic expression. Due to their abstract nature, there is a broad spectrum of emotion realization in artworks. These are subject to historical change and their analysis requires expertise in art history. In this article, we investigate which aspects of emotional expression can be detected by current (2025) vision language models (VLMs). We present a case study of three VLMs (Llava-Llama and two Qwen models) in which we ask these models four sets of questions of increasing complexity about artworks (general content, emotional content, expression of emotions, and emotion symbols) and carry out a qualitative expert evaluation. We find that the VLMs recognize the content of the images surprisingly well and often also which emotions they depict and how they are expressed. The models perform best for concrete images but fail for highly abstract or highly symbolic images. Reliable recognition of symbols remains fundamentally difficult. Furthermore, the models continue to exhibit the well-known LLM weakness of providing inconsistent answers to related questions.

📄 PDF Abstract BibTeX arXiv:2511.22929

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph

2024-09-17 · Yanbei Jiang, Krista A. Ehinger, Jey Han Lau

Exploring the narratives conveyed by fine-art paintings is a challenge in image captioning, where the goal is to generate descriptions that not only precisely represent the visual content but also offer a in-depth interp…

cross-modal alignmentImage CaptioningLanguage ModelingLanguage Modelling

Engaging with Children's Artwork in Mixed Visual-Ability Families

2024-07-26 · Arnavi Chheda-Kothary, Jacob O. Wobbrock, Jon E. Froehlich

We present two studies exploring how blind or low-vision (BLV) family members engage with their sighted children's artwork, strategies to support understanding and interpretation, and the potential role of technology, su…

Exploring the Synergy Between Vision-Language Pretraining and ChatGPT for Artwork Captioning: A Preliminary Study

2023-01-21 · FAPER Workshop - ICIAP 2023 2023 1 · Giovanna Castellano, Nicola Fanelli, Raffaele Scaringi, Gennaro Vessio

While AI techniques have enabled automated analysis and interpretation of visual content, generating meaningful captions for artworks presents unique challenges. These include understanding artistic intent, historical co…

Image CaptioningInformativenessMulti-Task LearningPrompt Engineering

Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models

2026-05-15 · Cheng Zhang, Yuer Liu, Zhiyu Zhou, Hongxia Xie 외 arxiv

Multimodal large language models (MLLMs) can produce fluent artwork emotion explanations, but they often suffer from attribute flooding: they enumerate many visible formal attributes without identifying which cues actual…

Exploring Content-based Artwork Recommendation with Metadata and Visual Features

2017-06-19 · Pablo Messina, Vicente Dominguez, Denis Parra, Christoph Trattner 외

Compared to other areas, artwork recommendation has received little attention, despite the continuous growth of the artwork market. Previous research has relied on ratings and metadata to make artwork recommendations, as…