Generalized Few-shot Semantic Segmentation
Training semantic segmentation models requires a large amount of finely annotated data, making it hard to quickly adapt to novel classes not satisfying this condition. Few-Shot Segmentation (FS-Seg) tackles this problem with many constraints. In this paper, we introduce a new benchmark, called Generalized Few-Shot Semantic Segmentation (GFS-Seg), to analyze the generalization ability of simultaneously segmenting the novel categories with very few examples and the base categories with sufficient examples. It is the first study showing that previous representative state-of-the-art FS-Seg methods fall short in GFS-Seg and the performance discrepancy mainly comes from the constrained setting of FS-Seg. To make GFS-Seg tractable, we set up a GFS-Seg baseline that achieves decent performance without structural change on the original model. Then, since context is essential for semantic segmentation, we propose the Context-Aware Prototype Learning (CAPL) that significantly improves performance by 1) leveraging the co-occurrence prior knowledge from support samples, and 2) dynamically enriching contextual information to the classifier, conditioned on the content of each query image. Both two contributions are experimentally shown to have substantial practical merit. Extensive experiments on Pascal-VOC and COCO manifest the effectiveness of CAPL, and CAPL generalizes well to FS-Seg by achieving competitive performance. Code is available at https://github.com/dvlab-research/GFS-Seg.
Code (1)
Tasks
Few-Shot Semantic SegmentationGeneralized Few-Shot Semantic SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark
Learning with limited labelled data is a challenging problem in various applications, including remote sensing. Few-shot semantic segmentation is one approach that can encourage deep learning models to learn from few lab…
Few-Shot Semantic SegmentationGeneralized Few-Shot Semantic SegmentationSegmentationSemantic SegmentationFew-shot 3D LiDAR Semantic Segmentation for Autonomous Driving
In autonomous driving, the novel objects and lack of annotations challenge the traditional 3D LiDAR semantic segmentation based on deep learning. Few-shot learning is a feasible way to solve these issues. However, curren…
Autonomous DrivingFew-Shot LearningFew-Shot Semantic SegmentationGeneralized Few-Shot Semantic Segmentation+4Applying ViT in Generalized Few-shot Semantic Segmentation
This paper explores the capability of ViT-based models under the generalized few-shot semantic segmentation (GFSS) framework. We conduct experiments with various combinations of backbone models, including ResNets and pre…
DecoderFew-Shot Semantic SegmentationGeneralized Few-Shot Semantic SegmentationSegmentation+1Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning
Generalized few-shot semantic segmentation was introduced to move beyond only evaluating few-shot segmentation models on novel classes to include testing their ability to remember base classes. While the current state-of…
AllFew-Shot Semantic SegmentationGeneralized Few-Shot Semantic SegmentationMeta-Learning+3A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation
The goal of generalized few-shot semantic segmentation (GFSS) is to recognize novel-class objects through training with a few annotated examples and the base-class model that learned the knowledge about the base classes.…
Few-Shot Semantic SegmentationGeneralized Few-Shot Semantic SegmentationInductive LearningSegmentation+1