Have Large Vision-Language Models Mastered Art History?

2024-09-05 · Ombretta Strafforello, Derya Soydaner, Michiel Willems, Anne-Sofie Maerten, Stefanie De Winter

The emergence of large Vision-Language Models (VLMs) has recently established new baselines in image classification across multiple domains. However, the performance of VLMs in the specific task of artwork classification, particularly art style classification of paintings - a domain traditionally mastered by art historians - has not been explored yet. Artworks pose a unique challenge compared to natural images due to their inherently complex and diverse structures, characterized by variable compositions and styles. Art historians have long studied the unique aspects of artworks, with style prediction being a crucial component of their discipline. This paper investigates whether large VLMs, which integrate visual and textual data, can effectively predict the art historical attributes of paintings. We conduct an in-depth analysis of four VLMs, namely CLIP, LLaVA, OpenFlamingo, and GPT-4o, focusing on zero-shot classification of art style, author and time period using two public benchmarks of artworks. Additionally, we present ArTest, a well-curated test set of artworks, including pivotal paintings studied by art historians.

📄 PDF Abstract BibTeX arXiv:2409.03521

Code (0)

등록된 구현이 없습니다.

Tasks

Classification image-classification Image Classificationzero-shot-classificationZero-Shot Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…

EstNLTK 1.6: Remastered Estonian NLP Pipeline

2020-05-01 · LREC 2020 5 · Sven Laur, Siim Orasmaa, Dage S{\"a}rg, Paul Tammo

The goal of the EstNLTK Python library is to provide a unified programming interface for natural language processing in Estonian. As such, previous versions of the library have been immensely successful both in academic …

Morphological Analysis

Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Models

2024-10-08 · Bozhou Li, Hao Liang, Yang Li, Fangcheng Fu 외

During the pretraining phase, large language models (LLMs) acquire vast amounts of knowledge from extensive text corpora. Nevertheless, in later stages such as fine-tuning and inference, the model may encounter knowledge…

Hallucination Overall - Test

MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models

2026-04-18 · Zhaokang Liao, Yingguo Gao, Yi Yang, Yongheng Hu 외 arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising approach to improve the reasoning abilities of Large Language Models (LLMs). Among RLVR algorithms, Group Relative Policy Optimization (GRP…

Reinforcement Learning

Visual Reasoning through Tool-supervised Reinforcement Learning

2026-04-21 · Qihua Dong, Gozde Sahin, Pei Wang, Zhaowei Cai 외 arxiv

In this paper, we investigate the problem of how to effectively master tool-use to solve complex visual reasoning tasks for Multimodal Large Language Models. To achieve that, we propose a novel Tool-supervised Reinforcem…

Reinforcement Learning Visual Reasoning

Boosting LLM Reasoning via Human-Inspired Reward Shaping

2026-02-04 · Wenze Lin, Zhen Yang, Xitai Jiang, Xiaoteng Ma 외 arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for enhancing reasoning in Large Language Models (LLMs). However, existing reward formulations typically treat exploration and con…