A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentation
In temporal action segmentation, Timestamp supervision requires only a handful of labelled frames per video sequence. For unlabelled frames, previous works rely on assigning hard labels, and performance rapidly collapses under subtle violations of the annotation assumptions. We propose a novel Expectation-Maximization (EM) based approach that leverages the label uncertainty of unlabelled frames and is robust enough to accommodate possible annotation errors. With accurate timestamp annotations, our proposed method produces SOTA results and even exceeds the fully-supervised setup in several metrics and datasets. When applied to timestamp annotations with missing action segments, our method presents stable performance. To further test our formulation's robustness, we introduce the new challenging annotation setup of Skip-tag supervision. This setup relaxes constraints and requires annotations of any fixed number of random frames in a video, making it more flexible than Timestamp supervision while remaining competitive.
Code (0)
등록된 구현이 없습니다.
Tasks
Action SegmentationTAGTemporal Action SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Temporal Action Segmentation from Timestamp Supervision
Temporal action segmentation approaches have been very successful recently. However, annotating videos with frame-wise labels to train such models is very expensive and time consuming. While weakly supervised methods tra…
Action SegmentationSegmentationTemporal Action SegmentationWeakly Supervised Action LocalizationTimestamp-Supervised Action Segmentation with Graph Convolutional Networks
We introduce a novel approach for temporal activity segmentation with timestamp supervision. Our main contribution is a graph convolutional network, which is learned in an end-to-end manner to exploit both frame features…
Action SegmentationSegmentationAction Recognition from Single Timestamp Supervision in Untrimmed Videos
Recognising actions in videos relies on labelled supervision during training, typically the start and end times of each action instance. This supervision is not only subjective, but also expensive to acquire. Weak video-…
Action RecognitionTemporal Action LocalizationTurning to a Teacher for Timestamp Supervised Temporal Action Segmentation
Temporal action segmentation in videos has drawn much attention recently. Timestamp supervision is a cost-effective way for this task. To obtain more information to optimize the model, the existing method generated pseud…
Action SegmentationModel OptimizationSegmentationTemporal Action SegmentationUnified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translation
This paper introduces a unified framework for video action segmentation via sequence to sequence (seq2seq) translation in a fully and timestamp supervised setup. In contrast to current state-of-the-art frame-level predic…
Action SegmentationDecoderTemporal Action SegmentationTranslation