PMVOS: Pixel-Level Matching-Based Video Object Segmentation
Semi-supervised video object segmentation (VOS) aims to segment arbitrary target objects in video when the ground truth segmentation mask of the initial frame is provided. Due to this limitation of using prior knowledge about the target object, feature matching, which compares template features representing the target object with input features, is an essential step. Recently, pixel-level matching (PM), which matches every pixel in template features and input features, has been widely used for feature matching because of its high performance. However, despite its effectiveness, the information used to build the template features is limited to the initial and previous frames. We address this issue by proposing a novel method-PM-based video object segmentation (PMVOS)-that constructs strong template features containing the information of all past frames. Furthermore, we apply self-attention to the similarity maps generated from PM to capture global dependencies. On the DAVIS 2016 validation set, we achieve new state-of-the-art performance among real-time methods (> 30 fps), with a J&F score of 85.6%. Performance on the DAVIS 2017 and YouTube-VOS validation sets is also impressive, with J&F scores of 74.0% and 68.2%, respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectOne-shot visual object segmentationSegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
RPM-Net: Robust Pixel-Level Matching Networks for Self-Supervised Video Object Segmentation
In this paper, we introduce a self-supervised approach for video object segmentation without human labeled data.Specifically, we present Robust Pixel-level Matching Net-works (RPM-Net), a novel deep architecture that mat…
ObjectSegmentationSemantic SegmentationVideo Object Segmentation+1Pixel-Level Bijective Matching for Video Object Segmentation
Semi-supervised video object segmentation (VOS) aims to track the designated objects present in the initial frame of a video at the pixel level. To fully exploit the appearance information of an object, pixel-level featu…
ObjectSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object Segmentation+1Pixel-Level Matching for Video Object Segmentation using Convolutional Neural Networks
We propose a novel video object segmentation algorithm based on pixel-level matching using Convolutional Neural Networks (CNN). Our network aims to distinguish the target area from the background on the basis of the pixe…
Feature CompressionObjectSemantic SegmentationSemi-Supervised Video Object Segmentation+3Joint-task Self-supervised Learning for Temporal Correspondence
This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner. Our learning process integrates two highly related tasks: tracking large image regions \emph{and} establishing fine-grai…
Object TrackingSelf-Supervised LearningSemi-Supervised Video Object SegmentationUnsupervised Video Object SegmentationLearning to Associate Every Segment for Video Panoptic Segmentation
Temporal correspondence - linking pixels or objects across frames - is a fundamental supervisory signal for the video models. For the panoptic understanding of dynamic scenes, we further extend this concept to every segm…
Panoptic SegmentationVideo Panoptic Segmentation