Continual Segmentation with Disentangled Objectness Learning and Class Recognition
Most continual segmentation methods tackle the problem as a per-pixel classification task. However, such a paradigm is very challenging, and we find query-based segmenters with built-in objectness have inherent advantages compared with per-pixel ones, as objectness has strong transfer ability and forgetting resistance. Based on these findings, we propose CoMasTRe by disentangling continual segmentation into two stages: forgetting-resistant continual objectness learning and well-researched continual classification. CoMasTRe uses a two-stage segmenter learning class-agnostic mask proposals at the first stage and leaving recognition to the second stage. During continual learning, a simple but effective distillation is adopted to strengthen objectness. To further mitigate the forgetting of old classes, we design a multi-label class distillation strategy suited for segmentation. We assess the effectiveness of CoMasTRe on PASCAL VOC and ADE20K. Extensive experiments show that our method outperforms per-pixel and query-based methods on both datasets. Code will be available at https://github.com/jordangong/CoMasTRe.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSegmentationSimilar Papers 제목 키워드 기반
Rethinking Query-based Transformer for Continual Image Segmentation
Class-incremental/Continual image segmentation (CIS) aims to train an image segmenter in stages, where the set of available categories differs at each stage. To leverage the built-in objectness of query-based transformer…
Continual LearningImage SegmentationSemantic SegmentationExploring Orthogonality in Open World Object Detection
Open world object detection aims to identify objects of unseen categories and incrementally recognize them once their annotations are provided. In distinction to the traditional paradigm that is limited to predefined…
Incremental LearningObjectobject-detectionObject Detection+1Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation
Incremental or continual learning has been extensively studied for image classification tasks to alleviate catastrophic forgetting, a phenomenon that earlier learned knowledge is forgotten when learning new concepts. For…
Class-Incremental Semantic SegmentationContinual Learningimage-classificationImage Classification+1Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation
Open-vocabulary panoptic segmentation remains hindered by two coupled issues: (i) mask selection bias, where objectness heads trained on closed vocabularies suppress masks of categories not observed in training, and (ii)…
Panoptic SegmentationImage ClassificationContinual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations
Deep neural networks suffer from the major limitation of catastrophic forgetting old tasks when learning new ones. In this paper we focus on class incremental continual learning in semantic segmentation, where new catego…
Continual LearningContinual Semantic SegmentationContrastive LearningDisjoint 10-1+6