Motion-inductive Self-supervised Object Discovery in Videos
In this paper, we consider the task of unsupervised object discovery in videos. Previous works have shown promising results via processing optical flows to segment objects. However, taking flow as input brings about two drawbacks. First, flow cannot capture sufficient cues when objects remain static or partially occluded. Second, it is challenging to establish temporal coherency from flow-only input, due to the missing texture information. To tackle these limitations, we propose a model for directly processing consecutive RGB frames, and infer the optical flow between any pair of frames using a layered representation, with the opacity channels being treated as the segmentation. Additionally, to enforce object permanence, we apply temporal consistency loss on the inferred masks from randomly-paired frames, which refer to the motions at different paces, and encourage the model to segment the objects even if they may not move at the current time point. Experimentally, we demonstrate superior performance over previous state-of-the-art methods on three public video segmentation datasets (DAVIS2016, SegTrackv2, and FBMS-59), while being computationally efficient by avoiding the overhead of computing optical flow as input.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectObject DiscoveryObject Discovery In VideosOptical Flow EstimationUnsupervised Object SegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Self-Supervision by Prediction for Object Discovery in Videos
Despite their irresistible success, deep learning algorithms still heavily rely on annotated data. On the other hand, unsupervised settings pose many challenges, especially about determining the right inductive bias in d…
Inductive BiasObjectObject DiscoveryObject Discovery In Videos+3Motion-Refined DINOSAUR for Unsupervised Multi-Object Discovery
Unsupervised multi-object discovery (MOD) aims to detect and localize distinct object instances in visual scenes without any form of human supervision. Recent approaches leverage object-centric learning (OCL) and motion …
Multi-object discoveryMotion SegmentationDIOD: Self-Distillation Meets Object Discovery
Instance segmentation demands substantial labeling resources. This has prompted increased interest to explore the object discovery task as an unsupervised alternative. In particular promising results were achieved in…
Instance SegmentationKnowledge DistillationObjectObject Discovery+1Sample, Crop, Track: Self-Supervised Mobile 3D Object Detection for Urban Driving LiDAR
Deep learning has led to great progress in the detection of mobile (i.e. movement-capable) objects in urban driving scenes in recent years. Supervised approaches typically require the annotation of large training sets; t…
3D Object DetectionObjectobject-detectionObject Detection+1Boosting Object Representation Learning via Motion and Object Continuity
Recent unsupervised multi-object detection models have shown impressive performance improvements, largely attributed to novel architectural inductive biases. Unfortunately, they may produce suboptimal object encodings fo…
Atari GamesObjectobject-detectionObject Detection+3