Attend and Enrich: Enhanced Visual Prompt for Zero-Shot Learning

2024-06-05 · Man Liu, Huihui Bai, Feng Li, Chunjie Zhang, Yunchao Wei, Tat-Seng Chua, Yao Zhao

Zero-shot learning (ZSL) endeavors to transfer knowledge from seen categories to recognize unseen categories, which mostly relies on the semantic-visual interactions between image and attribute tokens. Recently, prompt learning has emerged in ZSL and demonstrated significant potential as it allows the zero-shot transfer of diverse visual concepts to downstream tasks. However, current methods explore the fixed adaption of learnable prompt on seen domains, which makes them over-emphasize the primary visual features observed during training, limiting their generalization capabilities to unseen domains. In this work, we propose AENet, which endows semantic information into the visual prompt to distill semantic-enhanced prompt for visual representation enrichment, enabling effective knowledge transfer for ZSL. AENet comprises two key steps: 1) exploring the concept-harmonized tokens for the visual and attribute modalities, grounded on the modal-sharing token that represents consistent visual-semantic concepts; and 2) yielding semantic-enhanced prompt via the visual residual refinement unit with attribute consistency supervision. These are further integrated with primary visual features to attend to semantic-related information for visual enhancement, thus strengthening transferable ability. Experimental results on three benchmarks show that our AENet outperforms existing state-of-the-art ZSL methods. The code is provided in the zip file of supplementary materials.

📄 PDF Abstract BibTeX arXiv:2406.03032

Code (0)

등록된 구현이 없습니다.

Tasks

AttributeDomain Generalization Instruction Following Prompt EngineeringPrompt LearningTransfer Learning Zero-Shot Learning

Dual-Image Enhanced CLIP for Zero-Shot Anomaly Detection

2024-05-08 · Zhaoxiang Zhang, Hanqiu Deng, Jinan Bao, Xingyu Li

Image Anomaly Detection has been a challenging task in Computer Vision field. The advent of Vision-Language models, particularly the rise of CLIP-based frameworks, has opened new avenues for zero-shot anomaly detection. …

Anomaly Classification Anomaly Detection Test-time Adaptation zero-shot anomaly detection

GenCLIP: Generalizing CLIP Prompts for Zero-shot Anomaly Detection

2025-04-21 · Donghyeong Kim, Chaewon Park, Suhwan Cho, Hyeonjeong Lim 외

Zero-shot anomaly detection (ZSAD) aims to identify anomalies in unseen categories by leveraging CLIP's zero-shot capabilities to match text prompts with visual features. A key challenge in ZSAD is learning general promp…

Anomaly Detection Specificity zero-shot anomaly detection

TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision

2024-02-29 · Yunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li 외

Hierarchical text classification aims to categorize each document into a set of classes in a label taxonomy. Most earlier works focus on fully or semi-supervised methods that require a large amount of human annotated dat…

Classification text-classification Text Classification

Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval

2026-04-21 · Hang Cheng, Fanhe Dong, Long Zeng arxiv

This paper presents the first exploration of text-to-image diffusion models for zero-shot sketch-based 3D shape retrieval (ZS-SBSR). Existing sketch-based 3D shape retrieval methods struggle in zero-shot settings due to …

PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training

2026-04-01 · Weifu Fu, Jinyang Li, Bin-Bin Gao, Jialin Li 외 arxiv

Open-Set Object Detection (OSOD) enables recognition of novel categories beyond fixed classes but faces challenges in aligning text representations with complex visual concepts and the scarcity of image-text pairs for ra…

Zero-Shot Object Detection