Self-supervised Video Object Segmentation by Motion Grouping
Animals have evolved highly functional visual systems to understand motion, assisting perception even under complex environments. In this paper, we work towards developing a computer vision system able to segment objects by exploiting motion cues, i.e. motion segmentation. We make the following contributions: First, we introduce a simple variant of the Transformer to segment optical flow frames into primary objects and the background. Second, we train the architecture in a self-supervised manner, i.e. without using any manual annotations. Third, we analyze several critical components of our method and conduct thorough ablation studies to validate their necessity. Fourth, we evaluate the proposed architecture on public benchmarks (DAVIS2016, SegTrackv2, and FBMS59). Despite using only optical flow as input, our approach achieves superior or comparable results to previous state-of-the-art self-supervised methods, while being an order of magnitude faster. We additionally evaluate on a challenging camouflage dataset (MoCA), significantly outperforming the other self-supervised approaches, and comparing favourably to the top supervised approach, highlighting the importance of motion cues, and the potential bias towards visual appearance in existing video segmentation models.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion SegmentationObjectOptical Flow EstimationSemantic SegmentationUnsupervised Object SegmentationVideo Object SegmentationVideo SegmentationVideo Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Box Supervised Video Segmentation Proposal Network
Video Object Segmentation (VOS) has been targeted by various fully-supervised and self-supervised approaches. While fully-supervised methods demonstrate excellent results, self-supervised ones, which do not use pixel-lev…
Image SegmentationMotion CompensationObjectSegmentation+4MoDA: Leveraging Motion Priors from Videos for Advancing Unsupervised Domain Adaptation in Semantic Segmentation
Unsupervised domain adaptation (UDA) has been a potent technique to handle the lack of annotations in the target domain, particularly in semantic segmentation task. This study introduces a different UDA scenarios where t…
Domain AdaptationImage SegmentationObjectObject Discovery+7Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation
We propose a self-supervised spatio-temporal matching method, coined Motion-Aware Mask Propagation (MAMP), for video object segmentation. MAMP leverages the frame reconstruction task for training without the need for ann…
SegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object Segmentation+1Self-supervised Sparse to Dense Motion Segmentation
Observable motion in videos can give rise to the definition of objects moving with respect to the scene. The task of segmenting such moving objects is referred to as motion segmentation and is usually tackled either by a…
Motion SegmentationSegmentationMCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion
Autonomous systems, such as self-driving cars, rely on reliable semantic environment perception for decision making. Despite great advances in video semantic segmentation, existing approaches ignore important inductive b…
Decision MakingScene SegmentationSegmentationSelf-Driving Cars+2