Cross-Domain Few-Shot Semantic Segmentation via Doubly Matching Transformation
Cross-Domain Few-shot Semantic Segmentation (CD-FSS) aims to train generalized models that can segment classes from different domains with a few labeled images. Previous works have proven the effectiveness of feature transformation in addressing CD-FSS. However, they completely rely on support images for feature transformation, and repeatedly utilizing a few support images for each class may easily lead to overfitting and overlooking intra-class appearance differences. In this paper, we propose a Doubly Matching Transformation-based Network (DMTNet) to solve the above issue. Instead of completely relying on support images, we propose Self-Matching Transformation (SMT) to construct query-specific transformation matrices based on query images themselves to transform domain-specific query features into domain-agnostic ones. Calculating query-specific transformation matrices can prevent overfitting, especially for the meta-testing stage where only one or several images are used as support images to segment hundreds or thousands of images. After obtaining domain-agnostic features, we exploit a Dual Hypercorrelation Construction (DHC) module to explore the hypercorrelations between the query image with the foreground and background of the support image, based on which foreground and background prediction maps are generated and supervised, respectively, to enhance the segmentation result. In addition, we propose a Test-time Self-Finetuning (TSF) strategy to more accurately self-tune the query prediction in unseen domains. Extensive experiments on four popular datasets show that DMTNet achieves superior performance over state-of-the-art approaches. Code is available at https://github.com/ChenJiayi68/DMTNet.
Code (1)
Tasks
Cross-Domain Few-ShotFew-Shot Semantic SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Doubly Deformable Aggregation of Covariance Matrices for Few-shot Segmentation
Training semantic segmentation models with few annotated samples has great potential in various real-world applications. For the few-shot segmentation task, the main challenge is how to accurately measure the semantic co…
Few-Shot Semantic SegmentationSegmentationSemantic correspondenceSemantic SegmentationCross-Domain Few-Shot Semantic Segmentation
Few-shot semantic segmentation aims at learning to segment a novel object class with only a few annotated examples. Most existing methods consider a setting where base classes are sampled from the same domain as the no…
Cross-Domain Few-ShotFew-Shot Semantic SegmentationMeta-LearningSegmentation+2Remember the Difference: Cross-Domain Few-Shot Semantic Segmentation via Meta-Memory Transfer
Few-shot semantic segmentation intends to predict pixel level categories using only a few labeled samples. Existing few-shot methods focus primarily on the categories sampled from the same distribution. Nevertheless,…
Contrastive LearningCross-Domain Few-ShotFew-Shot LearningFew-Shot Semantic Segmentation+2APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation
Few-shot semantic segmentation (FSS) endeavors to segment unseen classes with only a few labeled samples. Current FSS methods are commonly built on the assumption that their training and application scenarios share simil…
Cross-Domain Few-ShotFew-Shot Semantic SegmentationSegmentationSemantic SegmentationCluster-to-adapt: Few Shot Domain Adaptation for Semantic Segmentation across Disjoint Labels
Domain adaptation for semantic segmentation across datasets consisting of the same categories has seen several recent successes. However, a more general scenario is when the source and target datasets correspond to non-o…
ClusteringDomain AdaptationSegmentationSemantic Segmentation