Video Instance Segmentation

2019-05-12 · ICCV 2019 10 · Linjie Yang, Yuchen Fan, Ning Xu

In this paper we present a new computer vision task, named video instance segmentation. The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain. To facilitate research on this new task, we propose a large-scale benchmark called YouTube-VIS, which consists of 2883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks. In addition, we propose a novel algorithm called MaskTrack R-CNN for this task. Our new method introduces a new tracking branch to Mask R-CNN to jointly perform the detection, segmentation and tracking tasks simultaneously. Finally, we evaluate the proposed method and several strong baselines on our new dataset. Experimental results clearly demonstrate the advantages of the proposed algorithm and reveal insight for future improvement. We believe the video instance segmentation task will motivate the community along the line of research for video understanding.

📄 PDF Abstract BibTeX arXiv:1905.04804

Code (6)

Epiphqny/VisTR 공식 구현 pytorch

JonathonLuiten/HOTA-metrics

JonathonLuiten/TrackEval

TranThienDat-Nguyen/VisionEvaluation

bbednarski9/MaskTrackRCNN_DeepSort pytorch

youtubevos/MaskTrackRCNN pytorch

Tasks

Instance Segmentation Segmentation Semantic Segmentation Video Instance SegmentationVideo Understanding

Methods 이 논문이 사용한 방법론

RPN A Region Proposal Network, or RPN, is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

RoIAlign Region of Interest Align, or RoIAlign, is an operation for extracting a small feature map from each RoI in detection and segmentation based tasks. It removes the harsh…

Mask R-CNN Mask R-CNN extends Faster R-CNN to solve instance segmentation tasks. It achieves this by adding a branch for predicting an…

Foundation Models for Amodal Video Instance Segmentation in Automated Driving

2024-09-21 · Jasmin Breitenstein, Franz Jünger, Andreas Bär, Tim Fingscheidt

In this work, we study amodal video instance segmentation for automated driving. Previous works perform amodal video instance segmentation relying on methods trained on entirely labeled video data with techniques borrowe…

Amodal Instance Segmentation Instance Segmentation Point Tracking Segmentation+2

Robust Online Video Instance Segmentation with Track Queries

2022-11-16 · Zitong Zhan, Daniel McKee, Svetlana Lazebnik

Recently, transformer-based methods have achieved impressive results on Video Instance Segmentation (VIS). However, most of these top-performing methods run in an offline manner by processing the entire video clip at onc…

Image Segmentation Instance Segmentation Multi-Object Tracking Object Tracking+5

FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matching

2025-05-19 · Alp Eren Sari, Paolo Favaro

We propose FlowCut, a simple and capable method for unsupervised video instance segmentation consisting of a three-stage framework to construct a high-quality video dataset with pseudo labels. To our knowledge, our work …

Instance Segmentation Segmentation Semantic Segmentation Video Instance Segmentation+2

Deep Learning Techniques for Video Instance Segmentation: A Survey

2023-10-19 · Chenhao Xu, Chang-Tsun Li, Yongjian Hu, Chee Peng Lim 외

Video instance segmentation, also known as multi-object tracking and segmentation, is an emerging computer vision research area introduced in 2019, aiming at detecting, segmenting, and tracking instances in videos simult…

Action Recognition Deep Learning Instance Segmentation Multi-Object Tracking+7

End-to-End Video Instance Segmentation with Transformers

2020-11-30 · CVPR 2021 1 · Yuqing Wang, Zhaoliang Xu, Xinlong Wang, Chunhua Shen 외

Video instance segmentation (VIS) is the task that requires simultaneously classifying, segmenting and tracking object instances of interest in video. Recent methods typically develop sophisticated pipelines to tackle th…

Instance Segmentation Segmentation Semantic Segmentation Video Instance Segmentation+1