Similarity Maps for Self-Training Weakly-Supervised Phrase Grounding
A phrase grounding model receives an input image and a text phrase and outputs a suitable localization map. We present an effective way to refine a phrase ground model by considering self-similarity maps extracted from the latent representation of the model's image encoder. Our main insights are that these maps resemble localization maps and that by combining such maps, one can obtain useful pseudo-labels for performing self-training. Our results surpass, by a large margin, the state-of-the-art in weakly supervised phrase grounding. A similar gap in performance is obtained for a recently proposed downstream task called WWbL, in which the input image is given without any text. Our code is available as supplementary.
Code (1)
Tasks
Phrase GroundingSimilar Papers 제목 키워드 기반
Unsupervised Object Localization with Representer Point Selection
We propose a novel unsupervised object localization method that allows us to explain the predictions of the model by utilizing self-supervised pre-trained models without additional finetuning. Existing unsupervised and s…
ObjectObject LocalizationUnsupervised Object LocalizationGenerating Self-Guided Dense Annotations for Weakly Supervised Semantic Segmentation
Learning semantic segmentation models under image-level supervision is far more challenging than under fully supervised setting. Without knowing the exact pixel-label correspondence, most weakly-supervised methods rely o…
SegmentationSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationA Self-Training Framework Based on Multi-Scale Attention Fusion for Weakly Supervised Semantic Segmentation
Weakly supervised semantic segmentation (WSSS) based on image-level labels is challenging since it is hard to obtain complete semantic regions. To address this issue, we propose a self-training method that utilizes fused…
DenoisingSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationSelf-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut
Transformers trained with self-supervised learning using self-distillation loss (DINO) have been shown to produce attention maps that highlight salient foreground objects. In this paper, we demonstrate a graph-based appr…
Objectobject-detectionObject DetectionObject Discovery+5MapRF: Weakly Supervised Online HD Map Construction via NeRF-Guided Self-Training
Autonomous driving systems benefit from high-definition (HD) maps that provide critical information about road infrastructure. The online construction of HD maps offers a scalable approach to generate local maps from on-…
Autonomous Driving