Learning Video Object Segmentation from Static Images
Inspired by recent advances of deep learning in instance segmentation and object tracking, we introduce video object segmentation problem as a concept of guided instance segmentation. Our model proceeds on a per-frame basis, guided by the output of the previous frame towards the object of interest in the next frame. We demonstrate that highly accurate object segmentation in videos can be enabled by using a convnet trained with static images only. The key ingredient of our approach is a combination of offline and online learning strategies, where the former serves to produce a refined mask from the previous frame estimate and the latter allows to capture the appearance of the specific object instance. Our method can handle different types of input annotations: bounding boxes and segments, as well as incorporate multiple annotated frames, making the system suitable for diverse applications. We obtain competitive results on three different datasets, independently from the type of input annotation.
Code (2)
Tasks
Instance SegmentationObjectObject TrackingSegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationVisual Object TrackingSimilar Papers 제목 키워드 기반
Instance Embedding Transfer to Unsupervised Video Object Segmentation
We propose a method for unsupervised video object segmentation by transferring the knowledge encapsulated in image-based instance embedding networks. The instance embedding network produces an embedding vector for each p…
ObjectOptical Flow EstimationSegmentationSemantic Segmentation+3DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping
We describe an unsupervised method to detect and segment portions of images of live scenes that, at some point in time, are seen moving as a coherent whole, which we refer to as objects. Our method first partitions the m…
Continual LearningObjectobject-detectionObject Detection+7HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Images
Existing state-of-the-art methods for Video Object Segmentation (VOS) learn low-level pixel-to-pixel correspondences between frames to propagate object masks across video. This requires a large amount of densely annotate…
ObjectSemantic SegmentationVideo Object SegmentationVideo Semantic SegmentationDynamic in Static: Hybrid Visual Correspondence for Self-Supervised Video Object Segmentation
Conventional video object segmentation (VOS) methods usually necessitate a substantial volume of pixel-level annotated video data for fully supervised learning. In this paper, we present HVC, a \textbf{h}ybrid static-dyn…
Semantic SegmentationVideo Object SegmentationVideo Semantic SegmentationSa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos
This work presents Sa2VA, the first unified model for dense grounded understanding of both images and videos. Unlike existing multi-modal large language models, which are often limited to specific modalities and tasks, S…
2kLanguage ModelingLanguage ModellingObject+6