WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation
This paper explores the properties of the plain Vision Transformer (ViT) for Weakly-supervised Semantic Segmentation (WSSS). The class activation map (CAM) is of critical importance for understanding a classification network and launching WSSS. We observe that different attention heads of ViT focus on different image areas. Thus a novel weight-based method is proposed to end-to-end estimate the importance of attention heads, while the self-attention maps are adaptively fused for high-quality CAM results that tend to have more complete objects. Besides, we propose a ViT-based gradient clipping decoder for online retraining with the CAM results to complete the WSSS task. We name this plain Transformer-based Weakly-supervised learning framework WeakTr. It achieves the state-of-the-art WSSS performance on standard benchmarks, i.e., 78.4% mIoU on the val set of PASCAL VOC 2012 and 50.3% mIoU on the val set of COCO 2014. Code is available at https://github.com/hustvl/WeakTr.
Code (1)
Tasks
DecoderSemantic SegmentationWeakly-supervised LearningWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
eX-ViT: A Novel eXplainable Vision Transformer for Weakly Supervised Semantic Segmentation
Recently vision transformer models have become prominent models for a range of vision tasks. These models, however, are usually opaque with weak feature interpretability. Moreover, there is no method currently built for …
AttributeSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationExploring Plain Vision Transformer Backbones for Object Detection
We explore the plain, non-hierarchical Vision Transformer (ViT) as a backbone network for object detection. This design enables the original ViT architecture to be fine-tuned for object detection without needing to redes…
Cross-Domain Few-Shot Object DetectionInstance SegmentationObjectobject-detection+1A Simple Vision Transformer for Weakly Semi-supervised 3D Object Detection
Advanced 3D object detection methods usually rely on large-scale, elaborately labeled datasets to achieve good performance. However, labeling the bounding boxes for the 3D objects is difficult and expensive. Although…
3D Object DetectionObjectobject-detectionObject DetectionTell Me Where to Look: Guided Attention Inference Network
Weakly supervised learning with only coarse labels can obtain visual explanations of deep neural network such as attention maps by back-propagating gradients. These attention maps are then available as priors for tasks s…
Object LocalizationSemantic SegmentationWeakly-supervised LearningAdversarial Normalization: I Can Visualize Everything (ICE)
Vision transformers use [CLS] tokens to predict image classes. Their explainability visualization has been studied using relevant information from [CLS] tokens or focusing on attention scores during self-attention. S…
Object DiscoveryObject LocalizationWeakly-Supervised Object Localization