Space-time Reinforcement Network for Video Object Segmentation
Recently, video object segmentation (VOS) networks typically use memory-based methods: for each query frame, the mask is predicted by space-time matching to memory frames. Despite these methods having superior performance, they suffer from two issues: 1) Challenging data can destroy the space-time coherence between adjacent video frames. 2) Pixel-level matching will lead to undesired mismatching caused by the noises or distractors. To address the aforementioned issues, we first propose to generate an auxiliary frame between adjacent frames, serving as an implicit short-temporal reference for the query one. Next, we learn a prototype for each video object and prototype-level matching can be implemented between the query and memory. The experiment demonstrated that our network outperforms the state-of-the-art method on the DAVIS 2017, achieving a J&F score of 86.4%, and attains a competitive result 85.0% on YouTube VOS 2018. In addition, our network exhibits a high inference speed of 32+ FPS.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectSemantic SegmentationVideo Object SegmentationVideo Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ALBA : Reinforcement Learning for Video Object Segmentation
We consider the challenging problem of zero-shot video object segmentation (VOS). That is, segmenting and tracking multiple moving objects within a video fully automatically, without any manual initialization. We treat t…
ObjectOne-shot visual object segmentationreinforcement-learningReinforcement Learning+6Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Time
We pose video object segmentation as spectral graph clustering in space and time, with one graph node for each pixel and edges forming local space-time neighborhoods. We claim that the strongest cluster in this video gra…
ClusteringGPUGraph ClusteringObject+6A Simple Video Segmenter by Tracking Objects Along Axial Trajectories
Video segmentation requires consistently segmenting and tracking objects over time. Due to the quadratic dependency on input size, directly applying self-attention to video segmentation with high-resolution input feature…
GPUObjectObject TrackingPanoptic Segmentation+5Unsupervised Learning of Object Keypoints for Perception and Control
The study of object representations in computer vision has primarily focused on developing representations that are useful for image classification, object detection, or semantic segmentation as downstream tasks. In this…
3D Action Recognitionimage-classificationImage ClassificationObject+6Towards Good Practices for Video Object Segmentation
Semi-supervised video object segmentation is an interesting yet challenging task in machine learning. In this work, we conduct a series of refinements with the propagation-based video object segmentation method and empir…
BIG-bench Machine LearningObjectOne-shot visual object segmentationSegmentation+4