Training Vision Transformers for Semi-Supervised Semantic Segmentation
We present S4Former a novel approach to training Vision Transformers for Semi-Supervised Semantic Segmentation (S4). At its core S4Former employs a Vision Transformer within a classic teacher-student framework and then leverages three novel technical ingredients: PatchShuffle as a parameter-free perturbation technique Patch-Adaptive Self-Attention (PASA) as a fine-grained feature modulation method and the innovative Negative Class Ranking (NCR) regularization loss. Based on these regularization modules aligned with Transformer-specific characteristics across the image input feature and output dimensions S4Former exploits the Transformer's ability to capture and differentiate consistent global contextual information in unlabeled images. Overall S4Former not only defines a new state of the art in S4 but also maintains a streamlined and scalable architecture. Being readily compatible with existing frameworks S4Former achieves strong improvements (up to 4.9%) on benchmarks like Pascal VOC 2012 COCO and Cityscapes with varying numbers of labeled data. The code is at https://github.com/JoyHuYY1412/S4Former.
Code (1)
Tasks
Semantic SegmentationSemi-Supervised Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Semi-Supervised Vision Transformers
We study the training of Vision Transformers for semi-supervised image classification. Transformers have recently demonstrated impressive performance on a multitude of supervised learning tasks. Surprisingly, we show Vis…
image-classificationImage ClassificationInductive BiasSemi-Supervised Image ClassificationSemi-supervised Vision Transformers at Scale
We study semi-supervised learning (SSL) for vision transformers (ViT), an under-explored topic despite the wide adoption of the ViT architectures to different tasks. To tackle this problem, we propose a new SSL pipeline,…
Inductive BiasSemi-Supervised Image ClassificationExploring Token-Level Augmentation in Vision Transformer for Semi-Supervised Semantic Segmentation
Semi-supervised semantic segmentation has witnessed remarkable advancements in recent years. However, existing algorithms are based on convolutional neural networks and directly applying them to Vision Transformers poses…
Data AugmentationDiversityImage AugmentationSemantic Segmentation+1On the Surprising Effectiveness of Transformers in Low-Labeled Video Recognition
Recently vision transformers have been shown to be competitive with convolution-based methods (CNNs) broadly across multiple vision tasks. The less restrictive inductive bias of transformers endows greater representation…
image-classificationImage ClassificationInductive BiasVideo Classification+1Lexico-semantic and affective modelling of Spanish poetry: A semi-supervised learning approach
Text classification tasks have improved substantially during the last years by the usage of transformers. However, the majority of researches focus on prose texts, with poetry receiving less attention, specially for Span…
SentenceSentence Embeddingstext-classificationText Classification