Self-Supervised Multi-Task Procedure Learning from Instructional Videos
We address the problem of unsupervised procedure learning from instructional videos of multiple tasks using Deep Neural Networks (DNNs). Unlike existing works, we assume that training videos come from multiple tasks without key-step annotations or grammars, and the goals are to classify a test video to the underlying task and to localize its key-steps. Our DNN learns task-dependent attention features from informative regions of each frame without ground-truth bounding boxes and learns to discover and localize key-steps without key-step annotations by using an unsupervised subset selection module as a teacher. It also learns to classify an input video using the discovered key-steps using a learnable key-step feature pooling mechanism that extracts and learns to combine key-step based features for task recognition. By experiments on two instructional video datasets, we show the effectiveness of our method for unsupervised localization of procedure steps and video classification.
Code (0)
등록된 구현이 없습니다.
Tasks
Procedure LearningVideo ClassificationSimilar Papers 제목 키워드 기반
StepFormer: Self-supervised Step Discovery and Localization in Instructional Videos
Instructional videos are an important resource to learn procedural tasks from human demonstrations. However, the instruction steps in such videos are typically short and sparse, with most of the video being irrelevant to…
DecoderP3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision
In this paper, we study the problem of procedure planning in instructional videos. Here, an agent must produce a plausible sequence of actions that can transform the environment from a given start to a desired goal state…
Collaborative Weakly Supervised Video Correlation Learning for Procedure-Aware Instructional Video Analysis
Video Correlation Learning (VCL), which aims to analyze the relationships between videos, has been widely studied and applied in various general video tasks. However, applying VCL to instructional videos is still quite c…
Action Quality AssessmentProcedure LearningUnsupervised Action Segmentation for Instructional Videos
In this paper we address the problem of automatically discovering atomic actions in unsupervised manner from instructional videos, which are rarely annotated with atomic actions. We present an unsupervised approach to le…
Action SegmentationSegmentationUnsupervised Action SegmentationUnsupervised Discovery of Actions in Instructional Videos
In this paper we address the problem of automatically discovering atomic actions in unsupervised manner from instructional videos. Instructional videos contain complex activities and are a rich source of information for …