Semantic Instance Meets Salient Object: Study on Video Semantic Salient Instance Segmentation
Focusing on only semantic instances that only salient in a scene gains more benefits for robot navigation and self-driving cars than looking at all objects in the whole scene. This paper pushes the envelope on salient regions in a video to decompose them into semantically meaningful components, namely, semantic salient instances. We provide the baseline for the new task of video semantic salient instance segmentation (VSSIS), that is, Semantic Instance - Salient Object (SISO) framework. The SISO framework is simple yet efficient, leveraging advantages of two different segmentation tasks, i.e. semantic instance segmentation and salient object segmentation to eventually fuse them for the final result. In SISO, we introduce a sequential fusion by looking at overlapping pixels between semantic instances and salient regions to have non-overlapping instances one by one. We also introduce a recurrent instance propagation to refine the shapes and semantic meanings of instances, and an identity tracking to maintain both the identity and the semantic meaning of instances over the entire video. Experimental results demonstrated the effectiveness of our SISO baseline, which can handle occlusions in videos. In addition, to tackle the task of VSSIS, we augment the DAVIS-2017 benchmark dataset by assigning semantic ground-truth for salient instance labels, obtaining SEmantic Salient Instance Video (SESIV) dataset. Our SESIV dataset consists of 84 high-quality video sequences with pixel-wisely per-frame ground-truth labels.
Code (0)
등록된 구현이 없습니다.
Tasks
Instance SegmentationRobot NavigationSegmentationSelf-Driving CarsSemantic SegmentationSimilar Papers 제목 키워드 기반
Associating Inter-Image Salient Instances for Weakly Supervised Semantic Segmentation
Effectively bridging between image level keyword annotations and corresponding image pixels is one of the main challenges in weakly supervised semantic segmentation. In this paper, we use an instance-level salient object…
Clusteringgraph partitioningImage-level Supervised Instance SegmentationInstance Segmentation+6Weakly-supervised Salient Instance Detection
Existing salient instance detection (SID) methods typically learn from pixel-level annotated datasets. In this paper, we present the first weakly-supervised approach to the SID problem. Although weak supervision has been…
Boundary DetectionObject LocalizationSaliency DetectionLearning to Detect Instance-level Salient Objects Using Complementary Image Labels
Existing salient instance detection (SID) methods typically learn from pixel-level annotated datasets. In this paper, we present the first weakly-supervised approach to the SID problem. Although weak supervision has been…
Boundary DetectionObject LocalizationSaliency DetectionSegGPT Meets Co-Saliency Scene
Co-salient object detection targets at detecting co-existed salient objects among a group of images. Recently, a generalist model for segmenting everything in context, called SegGPT, is gaining public attention. In view …
Co-Salient Object DetectionObjectobject-detectionObject Detection+1Instance-Level Salient Object Segmentation
Image saliency detection has recently witnessed rapid progress due to deep convolutional neural networks. However, none of the existing methods is able to identify object instances in the detected salient regions. In thi…
Instance SegmentationObjectSaliency DetectionSegmentation+1