Visual Adaptive Prompting for Compositional Zero-Shot Learning
Vision-Language Models (VLMs) have demonstrated impressive capabilities in learning joint representations of visual and textual data, making them powerful tools for tasks such as Compositional Zero-Shot Learning (CZSL). CZSL requires models to generalize to novel combinations of visual primitives-such as attributes and objects-that were not explicitly encountered during training. Recent works in prompting for CZSL have focused on modifying inputs for the text encoder, often using static prompts that do not change across varying visual contexts. However, these approaches struggle to fully capture varying visual contexts, as they focus on text adaptation rather than leveraging visual features for compositional reasoning. To address this, we propose Visual Adaptive Prompting System (VAPS) that leverages a learnable visual prompt repository and similarity-based retrieval mechanism within the framework of VLMs to bridge the gap between semantic and visual features. Our method introduces a dynamic visual prompt repository mechanism that selects the most relevant attribute and object prompts based on the visual features of the image. Our proposed system includes a visual prompt adapter that encourages the model to learn a more generalizable embedding space. Experiments on three CZSL benchmarks, across both closed and open-world scenarios, demonstrate state-of-the-art results.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeCompositional Zero-Shot LearningZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Prompting Language-Informed Distribution for Compositional Zero-Shot Learning
Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts, e.g., sliced tomatoes, where the model is learned only from the seen compositions, e.g., sliced potatoes and red tomato…
Compositional Zero-Shot LearningInformativenessZero-shot GeneralizationZero-Shot LearningPrompting Large Pre-trained Vision-Language Models For Compositional Concept Learning
This work explores the zero-shot compositional learning ability of large pre-trained vision-language models(VLMs) within the prompt-based learning framework and propose a model (\textit{PromptCompVL}) to solve the compos…
Zero-Shot LearningNovel Semantic Prompting for Zero-Shot Action Recognition
Zero-shot action recognition relies on transferring knowledge from vision-language models to unseen actions using semantic descriptions. While recent methods focus on temporal modeling or architectural adaptations to han…
Zero-Shot Action RecognitionAction UnderstandingPlug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognition
Large audio-language models (LALMs) exhibit strong zero-shot performance across speech tasks but struggle with speech emotion recognition (SER) due to weak paralinguistic modeling and limited cross-modal reasoning. We pr…
Speech Emotion RecognitionGIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning
Pre-trained vision-language models (VLMs) have achieved promising success in many fields, especially with prompt learning paradigm. In this work, we propose GIP-COL (Graph-Injected Soft Prompting for COmpositional Learni…
AttributeCompositional Zero-Shot LearningPrompt LearningZero-Shot Learning