Weakly Supervised Multiclass Video Segmentation
The desire of enabling computers to learn semantic concepts from large quantities of Internet videos has motivated increasing interests on semantic video understanding, while video segmentation is important yet challenging for understanding videos. The main difficulty of video segmentation arises from the burden of labeling training samples, making the problem largely unsolved. In this paper, we present a novel nearest neighbor-based label transfer scheme for weakly supervised video segmentation. Whereas previous weakly supervised video segmentation methods have been limited to the two-class case, our proposed scheme focuses on more challenging multiclass video segmentation, which finds a semantically meaningful label for every pixel in a video. Our scheme enjoys several favorable properties when compared with conventional methods. First, a weakly supervised hashing procedure is carried out to handle both metric and semantic similarity. Second, the proposed nearest neighbor-based label transfer algorithm effectively avoids overfitting caused by weakly supervised data. Third, a multi-video graph model is built to encourage smoothness between regions that are spatiotemporally adjacent and similar in appearance. We demonstrate the effectiveness of the proposed scheme by comparing it with several other state-of-the-art weakly supervised segmentation methods on one new Wild8 dataset and two other publicly available datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
SegmentationSemantic SimilaritySemantic Textual SimilarityVideo SegmentationVideo Semantic SegmentationVideo UnderstandingWeakly supervised segmentationSimilar Papers 제목 키워드 기반
J Regularization Improves Imbalanced Multiclass Segmentation
We propose a new loss formulation to further advance the multiclass segmentation of cluttered cells under weakly supervised conditions. We improve the separation of touching and immediate cells, obtaining sharp segmentat…
SegmentationLearning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
We address weakly-supervised video actor-action segmentation (VAAS), which extends general video object segmentation (VOS) to additionally consider action labels of the actors. The most successful methods on VOS synthesi…
Action SegmentationSegmentationSemantic SegmentationVideo Object Segmentation+1Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency
Weakly supervised instance segmentation reduces the cost of annotations required to train models. However, existing approaches which rely only on image-level class labels predominantly suffer from errors due to (a) parti…
Instance SegmentationRelation NetworkSegmentationSemantic Segmentation+1Weakly Supervised Semantic Segmentation using Web-Crawled Videos
We propose a novel algorithm for weakly supervised semantic segmentation based on image-level class labels only. In weakly supervised setting, it is commonly observed that trained model overly focuses on discriminative p…
image-classificationImage ClassificationSegmentationSemantic Segmentation+2Semi-Weakly-Supervised Learning of Complex Actions From Instructional Task Videos
We address the problem of action segmentation in instructional task videos with a small number of weakly-labeled training videos and a large number of unlabeled videos, which we refer to as Semi-Weakly-Supervised Lea…
Action SegmentationWeakly-supervised Learning