Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation
Existing studies in weakly supervised semantic segmentation (WSSS) have utilized class activation maps (CAMs) to localize the class objects. However, since a classification loss is insufficient for providing precise object regions, CAMs tend to be biased towards discriminative patterns (i.e., sparseness) and do not provide precise object boundary information (i.e., impreciseness). To resolve these limitations, we propose a novel framework (composed of MainNet and SupportNet.) that derives pixel-level self-supervision from given image-level supervision. In our framework, with the help of the proposed Regional Contrastive Module (RCM) and Multi-scale Attentive Module (MAM), MainNet is trained by self-supervision from the SupportNet. The RCM extracts two forms of self-supervision from SupportNet: (1) class region masks generated from the CAMs and (2) class-wise prototypes obtained from the features according to the class region masks. Then, every pixel-wise feature of the MainNet is trained by the prototype in a contrastive manner, sharpening the resulting CAMs. The MAM utilizes CAMs inferred at multiple scales from the SupportNet as self-supervision to guide the MainNet. Based on the dissimilarity between the multi-scale CAMs from MainNet and SupportNet, CAMs from the MainNet are trained to expand to the less-discriminative regions. The proposed method shows state-of-the-art WSSS performance both on the train and validation sets on the PASCAL VOC 2012 dataset. For reproducibility, code will be available publicly soon.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationSimilar Papers 제목 키워드 기반
Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation
We address the task of weakly-supervised few-shot image classification and segmentation, by leveraging a Vision Transformer (ViT) pretrained with self-supervision. Our proposed method takes token representations from the…
Few-Shot Image Classificationimage-classificationImage ClassificationPseudo Label+1Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation
Image-level weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. Most of advanced solutions exploit class activation map (CAM). However, CAMs can hardly serve as …
Data AugmentationSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationGenerating Self-Guided Dense Annotations for Weakly Supervised Semantic Segmentation
Learning semantic segmentation models under image-level supervision is far more challenging than under fully supervised setting. Without knowing the exact pixel-label correspondence, most weakly-supervised methods rely o…
SegmentationSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationBuilt-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
Pixel-level annotations are expensive and time consuming to obtain. Hence, weak supervision using only image tags could have a significant impact in semantic segmentation. Recently, CNN-based methods have proposed to fin…
SegmentationSemantic SegmentationTAGWeakly supervised Semantic Segmentation+1Weakly-Supervised Audio-Visual Segmentation
Audio-visual segmentation is a challenging task that aims to predict pixel-level masks for sound sources in a video. Previous work applied a comprehensive manually designed architecture with countless pixel-wise accurate…
Contrastive LearningSegmentation