Exploiting CLIP-based Multi-modal Approach for Artwork Classification and Retrieval
Given the recent advances in multimodal image pretraining where visual models trained with semantically dense textual supervision tend to have better generalization capabilities than those trained using categorical attributes or through unsupervised techniques, in this work we investigate how recent CLIP model can be applied in several tasks in artwork domain. We perform exhaustive experiments on the NoisyArt dataset which is a dataset of artwork images crawled from public resources on the web. On such dataset CLIP achieves impressive results on (zero-shot) classification and promising results in both artwork-to-artwork and description-to-artwork domain.
Code (0)
등록된 구현이 없습니다.
Tasks
Retrievalzero-shot-classificationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval
Analyzing digitized artworks presents unique challenges, requiring not only visual interpretation but also a deep understanding of rich artistic, contextual, and historical knowledge. We introduce ArtSeek, a multimodal f…
Visual Question AnsweringMultimodal ReasoningDraw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion
Digital art synthesis is receiving increasing attention in the multimedia community because of engaging the public with art effectively. Current digital art synthesis methods usually use single-modality inputs as guidanc…
DiversityCLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification
Existing computer vision research in artwork struggles with artwork's fine-grained attributes recognition and lack of curated annotated datasets due to their costly creation. To the best of our knowledge, we are one of t…
AttributeClassificationContrastive LearningFine-Grained Visual Recognition+2Harnessing Self-Supervised Features for Art Classification
Classifying artworks presents a significant challenge due to the complex interplay of fine-grained details and abstract features that condition the style or genre of an artwork. This paper presents a systematic investiga…
Have Large Vision-Language Models Mastered Art History?
The emergence of large Vision-Language Models (VLMs) has recently established new baselines in image classification across multiple domains. However, the performance of VLMs in the specific task of artwork classification…
Classificationimage-classificationImage Classificationzero-shot-classification+1