2by2: Weakly-Supervised Learning for Global Action Segmentation
This paper presents a simple yet effective approach for the poorly investigated task of global action segmentation, aiming at grouping frames capturing the same action across videos of different activities. Unlike the case of videos depicting all the same activity, the temporal order of actions is not roughly shared among all videos, making the task even more challenging. We propose to use activity labels to learn, in a weakly-supervised fashion, action representations suitable for global action segmentation. For this purpose, we introduce a triadic learning approach for video pairs, to ensure intra-video action discrimination, as well as inter-video and inter-activity action association. For the backbone architecture, we use a Siamese network based on sparse transformers that takes as input video pairs and determine whether they belong to the same activity. The proposed approach is validated on two challenging benchmark datasets: Breakfast and YouTube Instructions, outperforming state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action SegmentationWeakly-supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimized Global Perturbation Attacks For Brain Tumour ROI Extraction From Binary Classification Models
Deep learning techniques have greatly benefited computer-aided diagnostic systems. However, unlike other fields, in medical imaging, acquiring large fine-grained annotated datasets such as 3D tumour segmentation is chall…
Binary ClassificationDiagnosticSegmentationOn Evaluating Weakly Supervised Action Segmentation Methods
Action segmentation is the task of temporally segmenting every frame of an untrimmed video. Weakly supervised approaches to action segmentation, especially from transcripts have been of considerable interest to the compu…
Action SegmentationSegmentationTransformer based multiple instance learning for weakly supervised histopathology image segmentation
Hispathological image segmentation algorithms play a critical role in computer aided diagnosis technology. The development of weakly supervised segmentation algorithm alleviates the problem of medical image annotation th…
Image SegmentationMultiple Instance LearningSegmentationSemantic Segmentation+2Fast Weakly Supervised Action Segmentation Using Mutual Consistency
Action segmentation is the task of predicting the actions for each frame of a video. As obtaining the full annotation of videos for action segmentation is expensive, weakly supervised approaches that can learn only from …
Action SegmentationSegmentationWeakly Supervised Action Segmentation (Transcript)Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
We address weakly-supervised video actor-action segmentation (VAAS), which extends general video object segmentation (VOS) to additionally consider action labels of the actors. The most successful methods on VOS synthesi…
Action SegmentationSegmentationSemantic SegmentationVideo Object Segmentation+1