Center-guided Classifier for Semantic Segmentation of Remote Sensing Images
Compared with natural images, remote sensing images (RSIs) have the unique characteristic. i.e., larger intraclass variance, which makes semantic segmentation for remote sensing images more challenging. Moreover, existing semantic segmentation models for remote sensing images usually employ a vanilla softmax classifier, which has three drawbacks: (1) non-direct supervision for the pixel representations during training; (2) inadequate modeling ability of parametric softmax classifiers under large intraclass variance; and (3) opaque process of classification decision. In this paper, we propose a novel classifier (called CenterSeg) customized for RSI semantic segmentation, which solves the abovementioned problems with multiple prototypes, direct supervision under Grassmann manifold, and interpretability strategy. Specifically, for each class, our CenterSeg obtains local class centers by aggregating corresponding pixel features based on ground-truth masks, and generates multiple prototypes through hard attention assignment and momentum updating. In addition, we introduce the Grassmann manifold and constrain the joint embedding space of pixel features and prototypes based on two additional regularization terms. Especially, during the inference, CenterSeg can further provide interpretability to the model by restricting the prototype as a sample of the training set. Experimental results on three remote sensing segmentation datasets validate the effectiveness of the model. Besides the superior performance, CenterSeg has the advantages of simplicity, lightweight, compatibility, and interpretability. Code is available at https://github.com/xwmaxwma/rssegmentation.
Code (1)
Tasks
Hard AttentionSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MPerS: Dynamic MLLM MixExperts Perception-Guided Remote Sensing Scene Segmentation
The multimodal fusion of images and scene captions has been extensively explored and applied in various fields. However, when dealing with complex remote sensing (RS) scenes, existing studies have predominantly concentra…
Semantic SegmentationScene SegmentationSelf-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models
The Segment Anything Model (SAM) exhibits remarkable versatility and zero-shot learning abilities, owing largely to its extensive training data (SA-1B). Recognizing SAM's dependency on manual guidance given its category-…
Few-Shot Semantic SegmentationPrompt LearningSegmentationSemantic Segmentation+1Super-Resolution Domain Adaptation Networks for Semantic Segmentation via Pixel and Output Level Aligning
Recently, Unsupervised Domain Adaptation (UDA) has attracted increasing attention to address the domain shift problem in the semantic segmentation task. Although previous UDA methods have achieved promising performance, …
Domain AdaptationSegmentationSemantic SegmentationSuper-Resolution+1Understanding Imbalanced Semantic Segmentation Through Neural Collapse
A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phas…
3D Semantic SegmentationSegmentationSemantic SegmentationEarthMapper: A Tool Box for the Semantic Segmentation of Remote Sensing Imagery
Deep learning continues to push state-of-the-art performance for the semantic segmentation of color (i.e., RGB) imagery; however, the lack of annotated data for many remote sensing sensors (i.e. hyperspectral imagery (HS…
Deep LearningSegmentationSegmentation Of Remote Sensing ImagerySemantic Segmentation+1