Mask2Former for Video Instance Segmentation
We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline. In this report, we show universal image segmentation architectures trivially generalize to video segmentation by directly predicting 3D segmentation volumes. Specifically, Mask2Former sets a new state-of-the-art of 60.4 AP on YouTubeVIS-2019 and 52.6 AP on YouTubeVIS-2021. We believe Mask2Former is also capable of handling video semantic and panoptic segmentation, given its versatility in image segmentation. We hope this will make state-of-the-art video segmentation research more accessible and bring more attention to designing universal image and video segmentation architectures.
Code (6)
Tasks
Image SegmentationInstance SegmentationPanoptic SegmentationSegmentationSemantic SegmentationVideo Instance SegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Robust Online Video Instance Segmentation with Track Queries
Recently, transformer-based methods have achieved impressive results on Video Instance Segmentation (VIS). However, most of these top-performing methods run in an offline manner by processing the entire video clip at onc…
Image SegmentationInstance SegmentationMulti-Object TrackingObject Tracking+5TubeFormer-DeepLab: Video Mask Transformer
We present TubeFormer-DeepLab, the first attempt to tackle multiple core video segmentation tasks in a unified manner. Different video segmentation tasks (e.g., video semantic/instance/panoptic segmentation) are usually …
Panoptic SegmentationSegmentationVideo SegmentationVideo Semantic SegmentationMobileInst: Video Instance Segmentation on the Mobile
Video instance segmentation on mobile devices is an important yet very challenging edge AI problem. It mainly suffers from (1) heavy computation and memory costs for frame-by-frame pixel-level instance perception and (2)…
CPUDecoderInstance SegmentationSegmentation+2SeqFormer: Sequential Transformer for Video Instance Segmentation
In this work, we present SeqFormer for video instance segmentation. SeqFormer follows the principle of vision transformer that models instance relationships among video frames. Nevertheless, we observe that a stand-alone…
Instance SegmentationSemantic SegmentationVideo Instance SegmentationEnd-to-End Video Instance Segmentation with Transformers
Video instance segmentation (VIS) is the task that requires simultaneously classifying, segmenting and tracking object instances of interest in video. Recent methods typically develop sophisticated pipelines to tackle th…
Instance SegmentationSegmentationSemantic SegmentationVideo Instance Segmentation+1