One-Shot Instance Segmentation

2018-11-28 · Claudio Michaelis, Ivan Ustyuzhaninov, Matthias Bethge, Alexander S. Ecker

We tackle the problem of one-shot instance segmentation: Given an example image of a novel, previously unknown object category, find and segment all objects of this category within a complex scene. To address this challenging new task, we propose Siamese Mask R-CNN. It extends Mask R-CNN by a Siamese backbone encoding both reference image and scene, allowing it to target detection and segmentation towards the reference category. We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object categories works very well, targeting the detection network towards the reference category appears to be more difficult. Our work provides a first strong baseline for one-shot instance segmentation and will hopefully inspire further research into more powerful and flexible scene analysis algorithms. Code is available at: https://github.com/bethgelab/siamese-mask-rcnn

📄 PDF Abstract BibTeX arXiv:1811.11507

Code (3)

bethgelab/siamese-mask-rcnn 공식 구현 tf

ducminhkhoi/FAPIS pytorch

michaelisc/cluttered-omniglot tf

Tasks

Few-Shot Learning Few-Shot Object Detection Instance Segmentation Object Detection One-Shot Instance Segmentation One-Shot Learning One-Shot Object Detection Segmentation

Methods 이 논문이 사용한 방법론

RPN A Region Proposal Network, or RPN, is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

RoIAlign Region of Interest Align, or RoIAlign, is an operation for extracting a small feature map from each RoI in detection and segmentation based tasks. It removes the harsh…

Mask R-CNN Mask R-CNN extends Faster R-CNN to solve instance segmentation tasks. It achieves this by adding a branch for predicting an…

Zero-Shot Instance Segmentation

2021-04-14 · CVPR 2021 1 · Ye Zheng, JiaHong Wu, Yongqiang Qin, Faen Zhang 외

Deep learning has significantly improved the precision of instance segmentation with abundant labeled data. However, in many areas like medical and manufacturing, collecting sufficient data is extremely hard and labeling…

Instance Segmentation object-detection Object Detection Segmentation+3

FGN: Fully Guided Network for Few-Shot Instance Segmentation

2020-03-31 · CVPR 2020 6 · Zhibo Fan, Jin-Gang Yu, Zhihao Liang, Jiarong Ou 외

Few-shot instance segmentation (FSIS) conjoins the few-shot learning paradigm with general instance segmentation, which provides a possible way of tackling instance segmentation in the lack of abundant labeled data for t…

Few-shot Instance Segmentation Few-Shot Learning Instance Segmentation Segmentation+1

Bounding Box Embedding for Single Shot Person Instance Segmentation

2018-07-20 · Jacob Richeimer, Jonathan Mitchell

We present a bottom-up approach for the task of object instance segmentation using a single-shot model. The proposed model employs a fully convolutional network which is trained to predict class-wise segmentation masks a…

Instance Segmentation Object Segmentation Semantic Segmentation

Zero-shot Unsupervised Transfer Instance Segmentation

2023-04-27 · Gyungin Shin, Samuel Albanie, Weidi Xie

Segmentation is a core computer vision competency, with applications spanning a broad range of scientifically and economically valuable domains. To date, however, the prohibitive cost of annotation has limited the deploy…

Instance Segmentation Segmentation Semantic Segmentation

PanopticRecon: Leverage Open-vocabulary Instance Segmentation for Zero-shot Panoptic Reconstruction

2024-07-01 · Xuan Yu, Yili Liu, Chenrui Han, Sitong Mao 외

Panoptic reconstruction is a challenging task in 3D scene understanding. However, most existing methods heavily rely on pre-trained semantic segmentation models and known 3D object bounding boxes for 3D panoptic segmenta…

3D Panoptic Segmentation Instance Segmentation Panoptic Segmentation Scene Understanding+3