Video Object Segmentation using Space-Time Memory Networks
We propose a novel solution for semi-supervised video object segmentation. By the nature of the problem, available cues (e.g. video frame(s) with object masks) become richer with the intermediate predictions. However, the existing methods are unable to fully exploit this rich source of information. We resolve the issue by leveraging memory networks and learn to read relevant information from all available sources. In our framework, the past frames with object masks form an external memory, and the current frame as the query is segmented using the mask information in the memory. Specifically, the query and the memory are densely matched in the feature space, covering all the space-time pixel locations in a feed-forward fashion. Contrast to the previous approaches, the abundant use of the guidance information allows us to better handle the challenges such as appearance changes and occlussions. We validate our method on the latest benchmark sets and achieved the state-of-the-art performance (overall score of 79.4 on Youtube-VOS val set, J of 88.7 and 79.2 on DAVIS 2016/2017 val set respectively) while having a fast runtime (0.16 second/frame on DAVIS 2016 val set).
Code (3)
Tasks
Interactive Video Object SegmentationObjectOne-shot visual object segmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Space-time Reinforcement Network for Video Object Segmentation
Recently, video object segmentation (VOS) networks typically use memory-based methods: for each query frame, the mask is predicted by space-time matching to memory frames. Despite these methods having superior performanc…
ObjectSemantic SegmentationVideo Object SegmentationVideo Semantic SegmentationTowards Good Practices for Video Object Segmentation
Semi-supervised video object segmentation is an interesting yet challenging task in machine learning. In this work, we conduct a series of refinements with the propagation-based video object segmentation method and empir…
BIG-bench Machine LearningObjectOne-shot visual object segmentationSegmentation+4Fast Video Object Segmentation using the Global Context Module
We developed a real-time, high-quality semi-supervised video object segmentation algorithm. Its accuracy is on par with the most accurate, time-consuming online-learning model, while its speed is similar to the fastest t…
ObjectSegmentationSemantic SegmentationSemi-Supervised Video Object Segmentation+3Kernelized Memory Network for Video Object Segmentation
Semi-supervised video object segmentation (VOS) is a task that involves predicting a target object in a video when the ground truth segmentation mask of the target object is given in the first frame. Recently, space-time…
ObjectSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object Segmentation+1Multi-Object Tracking and Segmentation with a Space-Time Memory Network
We propose a method for multi-object tracking and segmentation based on a novel memory-based mechanism to associate tracklets. The proposed tracker, MeNToS, addresses particularly the long-term data association problem, …
Instance SegmentationMulti-Object TrackingMulti-Object Tracking and SegmentationMultiple Object Tracking+9