Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification
Text-to-image person re-identification (ReID) aims to search for images containing a person of interest using textual descriptions. However, due to the significant modality gap and the large intra-class variance in textual descriptions, text-to-image ReID remains a challenging problem. Accordingly, in this paper, we propose a Semantically Self-Aligned Network (SSAN) to handle the above problems. First, we propose a novel method that automatically extracts semantically aligned part-level features from the two modalities. Second, we design a multi-view non-local network that captures the relationships between body parts, thereby establishing better correspondences between body parts and noun phrases. Third, we introduce a Compound Ranking (CR) loss that makes use of textual descriptions for other images of the same identity to provide extra supervision, thereby effectively reducing the intra-class variance in textual features. Finally, to expedite future research in text-to-image ReID, we build a new database named ICFG-PEDES. Extensive experiments demonstrate that SSAN outperforms state-of-the-art approaches by significant margins. Both the new ICFG-PEDES database and the SSAN code are available at https://github.com/zifyloo/SSAN.
Code (1)
Tasks
Image RetrievalPerson Re-IdentificationText based Person RetrievalText-based Person Retrieval with Noisy CorrespondenceSimilar Papers 제목 키워드 기반
Densely Semantically Aligned Person Re-Identification
We propose a densely semantically aligned person re-identification framework. It fundamentally addresses the body misalignment problem caused by pose/viewpoint variations, imperfect person detection, occlusion, etc. By l…
Human DetectionPerson Re-IdentificationTowards Transformer-Based Aligned Generation with Self-Coherence Guidance
We introduce a novel, training-free approach for enhancing alignment in Transformer-based Text-Guided Diffusion Models (TGDMs). Existing TGDMs often struggle to generate semantically aligned images, particularly when dea…
AttributeDenoisingWhat You See is What You Read? Improving Text-Image Alignment Evaluation
Automatically determining whether a text and a corresponding image are semantically aligned is a significant challenge for vision-language models, with applications in generative text-to-image and image-to-text tasks. In…
Image GenerationImage to textQuestion AnsweringQuestion Generation+6Emphasizing Complementary Samples for Non-literal Cross-modal Retrieval
Existing cross-modal retrieval methods assume a straightforward relationship where images and text contain portrayals or mentions of the same objects. In contrast, real-world image-text pairs (e.g. an image and its capti…
Cross-Modal RetrievalRetrievalSemantic SimilaritySemantic Textual SimilaritySemantics-Aligned Representation Learning for Person Re-identification
Person re-identification (reID) aims to match person images to retrieve the ones with the same identity. This is a challenging task, as the images to be matched are generally semantically misaligned due to the diversity …
DecoderPerson Re-IdentificationRepresentation LearningTexture Synthesis+1