PM-VIS+: High-Performance Video Instance Segmentation without Video Annotation
Video instance segmentation requires detecting, segmenting, and tracking objects in videos, typically relying on costly video annotations. This paper introduces a method that eliminates video annotations by utilizing image datasets. The PM-VIS algorithm is adapted to handle both bounding box and instance-level pixel annotations dynamically. We introduce ImageNet-bbox to supplement missing categories in video datasets and propose the PM-VIS+ algorithm to adjust supervision based on annotation types. To enhance accuracy, we use pseudo masks and semi-supervised optimization techniques on unannotated video data. This method achieves high video instance segmentation performance without manual video annotations, offering a cost-effective solution and new perspectives for video instance segmentation applications. The code will be available in https://github.com/ldknight/PM-VIS-plus
Code (1)
Tasks
Instance SegmentationSegmentationSemantic SegmentationVideo Instance SegmentationSimilar Papers 제목 키워드 기반
MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Training
We propose MinVIS, a minimal video instance segmentation (VIS) framework that achieves state-of-the-art VIS performance with neither video-based architectures nor training procedures. By only training a query-based image…
Instance SegmentationSegmentationSemantic SegmentationVideo Instance SegmentationSeqFormer: Sequential Transformer for Video Instance Segmentation
In this work, we present SeqFormer for video instance segmentation. SeqFormer follows the principle of vision transformer that models instance relationships among video frames. Nevertheless, we observe that a stand-alone…
Instance SegmentationSemantic SegmentationVideo Instance SegmentationVideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation
Existing approaches to unsupervised video instance segmentation typically rely on motion estimates and experience difficulties tracking small or divergent motions. We present VideoCutLER, a simple method for unsupervised…
Instance SegmentationOptical Flow EstimationSegmentationSemantic Segmentation+3End-to-End Video Instance Segmentation with Transformers
Video instance segmentation (VIS) is the task that requires simultaneously classifying, segmenting and tracking object instances of interest in video. Recent methods typically develop sophisticated pipelines to tackle th…
Instance SegmentationSegmentationSemantic SegmentationVideo Instance Segmentation+1Mask2Former for Video Instance Segmentation
We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline. In this report, we show universal image segmentati…
Image SegmentationInstance SegmentationPanoptic SegmentationSegmentation+4