Exploiting the Textual Potential from Vision-Language Pre-training for Text-based Person Search
Text-based Person Search (TPS), is targeted on retrieving pedestrians to match text descriptions instead of query images. Recent Vision-Language Pre-training (VLP) models can bring transferable knowledge to downstream TPS tasks, resulting in more efficient performance gains. However, existing TPS methods improved by VLP only utilize pre-trained visual encoders, neglecting the corresponding textual representation and breaking the significant modality alignment learned from large-scale pre-training. In this paper, we explore the full utilization of textual potential from VLP in TPS tasks. We build on the proposed VLP-TPS baseline model, which is the first TPS model with both pre-trained modalities. We propose the Multi-Integrity Description Constraints (MIDC) to enhance the robustness of the textual modality by incorporating different components of fine-grained corpus during training. Inspired by the prompt approach for zero-shot classification with VLP models, we propose the Dynamic Attribute Prompt (DAP) to provide a unified corpus of fine-grained attributes as language hints for the image modality. Extensive experiments show that our proposed TPS framework achieves state-of-the-art performance, exceeding the previous best method by a margin.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributePerson SearchText based Person Searchzero-shot-classificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Exploiting Category Names for Few-Shot Classification with Vision-Language Models
Vision-language foundation models pretrained on large-scale data provide a powerful tool for many visual understanding tasks. Notably, many vision-language models build two encoders (visual and textual) that can map two …
ClassificationFew-Shot Image Classificationimage-classificationImage ClassificationZero-Shot Temporal Action Localization Through Textual Guidance
Zero-shot temporal action localization (ZS-TAL) consists of classifying and localizing actions in untrimmed videos, where action classes are unseen at training time. Existing work uses Vision and Language Models (VLMs), …
Temporal Action LocalizationAction ClassificationExploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere
Vision-language models (VLMs) as foundation models have significantly enhanced performance across a wide range of visual and textual tasks, without requiring large-scale training from scratch for downstream tasks. Howeve…
Uncertainty QuantificationRewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation
Vision-Language Pre-training has demonstrated its remarkable zero-shot recognition ability and potential to learn generalizable visual representations from language supervision. Taking a step ahead, language-supervised s…
SegmentationSemantic SegmentationZero-Shot LearningHarnessing Large Language Models for Training-free Video Anomaly Detection
Video anomaly detection (VAD) aims to temporally locate abnormal events in a video. Existing works mostly rely on training deep models to learn the distribution of normality with either video-level supervision, one-class…
Anomaly DetectionVideo Anomaly Detection