Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
This paper addresses a new problem of weakly-supervised online action segmentation in instructional videos. We present a framework to segment streaming videos online at test time using Dynamic Programming and show its advantages over greedy sliding window approach. We improve our framework by introducing the Online-Offline Discrepancy Loss (OODL) to encourage the segmentation results to have a higher temporal consistency. Furthermore, only during training, we exploit frame-wise correspondence between multiple views as supervision for training weakly-labeled instructional videos. In particular, we investigate three different multi-view inference techniques to generate more accurate frame-wise pseudo ground-truth with no additional annotation cost. We present results and ablation studies on two benchmark multi-view datasets, Breakfast and IKEA ASM. Experimental results show efficacy of the proposed methods both qualitatively and quantitatively in two domains of cooking and assembly.
Code (0)
등록된 구현이 없습니다.
Tasks
Action SegmentationSegmentationSimilar Papers 제목 키워드 기반
Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation
End-to-end weakly supervised semantic segmentation aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branc…
ClassificationSegmentationSemantic SegmentationWeakly supervised segmentation+2On Evaluating Weakly Supervised Action Segmentation Methods
Action segmentation is the task of temporally segmenting every frame of an untrimmed video. Weakly supervised approaches to action segmentation, especially from transcripts have been of considerable interest to the compu…
Action SegmentationSegmentationLearning from Video and Text via Large-Scale Discriminative Clustering
Discriminative clustering has been successfully applied to a number of weakly-supervised learning tasks. Such applications include person and action recognition, text-to-video alignment, object co-segmentation and coloca…
Action RecognitionClusteringTemporal Action LocalizationVideo Alignment+3CG-fusion CAM: Online segmentation of laser-induced damage on large-aperture optics
Online segmentation of laser-induced damage on large-aperture optics in high-power laser facilities is challenged by complicated damage morphology, uneven illumination and stray light interference. Fully supervised seman…
SegmentationSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationWOAD: Weakly Supervised Online Action Detection in Untrimmed Videos
Online action detection in untrimmed videos aims to identify an action as it happens, which makes it very important for real-time applications. Previous methods rely on tedious annotations of temporal action boundaries f…
Action DetectionAction RecognitionOnline Action Detection