Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation

2019-12-10 · CVPR 2020 6 · Gedas Bertasius, Lorenzo Torresani

We introduce a method for simultaneously classifying, segmenting and tracking object instances in a video sequence. Our method, named MaskProp, adapts the popular Mask R-CNN to video by adding a mask propagation branch that propagates frame-level object instance masks from each video frame to all the other frames in a video clip. This allows our system to predict clip-level instance tracks with respect to the object instances segmented in the middle frame of the clip. Clip-level instance tracks generated densely for each frame in the sequence are finally aggregated to produce video-level object instance segmentation and classification. Our experiments demonstrate that our clip-level instance segmentation makes our approach robust to motion blur and object occlusions in video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset, outperforming the ICCV 2019 video instance segmentation challenge winner despite being much simpler and using orders of magnitude less labeled data (1.3M vs 1B images and 860K vs 14M bounding boxes).

📄 PDF Abstract BibTeX arXiv:1912.04573

Code (0)

등록된 구현이 없습니다.

Tasks

Instance SegmentationObjectSegmentation Semantic Segmentation Video Instance Segmentation

Methods 이 논문이 사용한 방법론

RPN A Region Proposal Network, or RPN, is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

RoIAlign Region of Interest Align, or RoIAlign, is an operation for extracting a small feature map from each RoI in detection and segmentation based tasks. It removes the harsh…

Mask R-CNN Mask R-CNN extends Faster R-CNN to solve instance segmentation tasks. It achieves this by adding a branch for predicting an…

End-to-End Video Instance Segmentation with Transformers

2020-11-30 · CVPR 2021 1 · Yuqing Wang, Zhaoliang Xu, Xinlong Wang, Chunhua Shen 외

Video instance segmentation (VIS) is the task that requires simultaneously classifying, segmenting and tracking object instances of interest in video. Recent methods typically develop sophisticated pipelines to tackle th…

Instance Segmentation Segmentation Semantic Segmentation Video Instance Segmentation+1

Video Instance Segmentation by Instance Flow Assembly

2021-10-20 · Xiang Li, Jinglu Wang, Xiao Li, Yan Lu

Instance segmentation is a challenging task aiming at classifying and segmenting all object instances of specific classes. While two-stage box-based methods achieve top performances in the image domain, they cannot easil…

Instance Segmentation Object Object Localization Semantic Segmentation+1

Deformable VisTR: Spatio temporal deformable attention for video instance segmentation

2022-03-12 · Sudhir Yarram, Jialian Wu, Pan Ji, Yi Xu 외

Video instance segmentation (VIS) task requires classifying, segmenting, and tracking object instances over all frames in a video clip. Recently, VisTR has been proposed as end-to-end transformer-based VIS framework, whi…

GPU Instance Segmentation Semantic Segmentation Video Instance Segmentation

Instance-Level Video Segmentation From Object Tracks

2016-06-01 · CVPR 2016 6 · Guillaume Seguin, Piotr Bojanowski, Remi Lajugie, Ivan Laptev

We address the problem of segmenting multiple object instances in complex videos. Our method does not require manual pixel-level annotation for training, and relies instead on readily-available object detectors or visual…

Clustering Object Object Tracking Segmentation+4

UVIS: Unsupervised Video Instance Segmentation

2024-06-11 · Shuaiyi Huang, Saksham Suri, Kamal Gupta, Sai Saketh Rambhatla 외

Video instance segmentation requires classifying, segmenting, and tracking every object across video frames. Unlike existing approaches that rely on masks, boxes, or category labels, we propose UVIS, a novel Unsupervised…

Instance Segmentation Language Modelling Pseudo Label Segmentation+2