Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences
Understanding the structure of complex activities in untrimmed videos is a challenging task in the area of action recognition. One problem here is that this task usually requires a large amount of hand-annotated minute- or even hour-long video data, but annotating such data is very time consuming and can not easily be automated or scaled. To address this problem, this paper proposes an approach for the unsupervised learning of actions in untrimmed video sequences based on a joint visual-temporal embedding space. To this end, we combine a visual embedding based on a predictive U-Net architecture with a temporal continuous function. The resulting representation space allows detecting relevant action clusters based on their visual as well as their temporal appearance. The proposed method is evaluated on three standard benchmark datasets, Breakfast Actions, INRIA YouTube Instructional Videos, and 50 Salads. We show that the proposed approach is able to provide a meaningful visual and temporal embedding out of the visual cues present in contiguous video frames and is suitable for the task of unsupervised temporal segmentation of actions.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionAction SegmentationTemporal LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities
Action recognition and detection in the context of long untrimmed video sequences has seen an increased attention from the research community. However, annotation of complex activities is usually time consuming and chall…
Action RecognitionAction SegmentationUnsupervised Feature Learning of Human Actions as Trajectories in Pose Embedding Manifold
An unsupervised human action modeling framework can provide useful pose-sequence representation, which can be utilized in a variety of pose analysis applications. In this work we propose a novel temporal pose-sequence mo…
Action RecognitionFine-grained Action RecognitionRepresentation LearningTemporal Action LocalizationI2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification
Despite the tremendous progress in zero-shot learning(ZSL), the majority of existing methods still rely on human-annotated attributes, which are difficult to annotate and scale. An unsupervised alternative is to represen…
Generalized Zero-Shot Learningimage-classificationImage ClassificationWord Embeddings+2Retrieving and Highlighting Action with Spatiotemporal Reference
In this paper, we present a framework that jointly retrieves and spatiotemporally highlights actions in videos by enhancing current deep cross-modal retrieval methods. Our work takes on the novel task of action highlight…
Action RecognitionCross-Modal RetrievalExplainable ModelsRetrieval+3Predicting the Future: A Jointly Learnt Model for Action Anticipation
Inspired by human neurological structures for action anticipation, we present an action anticipation model that enables the prediction of plausible future actions by forecasting both the visual and temporal future. In co…
Action AnticipationGenerative Adversarial Network