RVOS: End-to-End Recurrent Network for Video Object Segmentation
Multiple object video object segmentation is a challenging task, specially for the zero-shot case, when no object mask is given at the initial frame and the model has to find the objects to be segmented along the sequence. In our work, we propose a Recurrent network for multiple object Video Object Segmentation (RVOS) that is fully end-to-end trainable. Our model incorporates recurrence on two different domains: (i) the spatial, which allows to discover the different object instances within a frame, and (ii) the temporal, which allows to keep the coherence of the segmented objects along time. We train RVOS for zero-shot video object segmentation and are the first ones to report quantitative results for DAVIS-2017 and YouTube-VOS benchmarks. Further, we adapt RVOS for one-shot video object segmentation by using the masks obtained in previous time steps as inputs to be processed by the recurrent module. Our model reaches comparable results to state-of-the-art techniques in YouTube-VOS benchmark and outperforms all previous video object segmentation methods not using online learning in the DAVIS-2017 benchmark. Moreover, our model achieves faster inference runtimes than previous methods, reaching 44ms/frame on a P100 GPU.
Code (1)
Tasks
GPUObjectOne-shot visual object segmentationSegmentationSemi-Supervised Video Object SegmentationUnsupervised Video Object SegmentationVideo Object SegmentationZero-Shot Video Object SegmentationSimilar Papers 제목 키워드 기반
Few-Shot Referring Video Single- and Multi-Object Segmentation via Cross-Modal Affinity with Instance Sequence Matching
Referring video object segmentation (RVOS) aims to segment objects in videos guided by natural language descriptions. We propose FS-RVOS, a Transformer-based model with two key components: a cross-modal affinity module a…
ObjectReferring Video Object SegmentationSegmentationSemantic Segmentation+21st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation
The recent transformer-based models have dominated the Referring Video Object Segmentation (RVOS) task due to the superior performance. Most prior works adopt unified DETR framework to generate segmentation masks in quer…
ObjectReferring Video Object SegmentationSegmentationSemantic Segmentation+2Show Me When and Where: Towards Referring Video Object Segmentation in the Wild
Referring video object segmentation (RVOS) has recently generated great popularity in computer vision due to its widespread applications. Existing RVOS setting contains elaborately trimmed videos, with text-referred obje…
Referring Video Object SegmentationURVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark
We propose a unified referring video object segmentation network (URVOS). URVOS takes a video and a referring expression as inputs, and estimates the {object masks} referred by the given language expression in the whole …
ObjectOne-shot visual object segmentationReferring ExpressionReferring Expression Segmentation+5Long-RVOS: A Comprehensive Benchmark for Long-term Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to identify, track and segment the objects in a video based on language descriptions, which has received great attention in recent years. However, existing datasets remain …
Referring Video Object SegmentationSemantic SegmentationVideo Object SegmentationVideo Semantic Segmentation