Weakly-Supervised Multi-Person Action Recognition in 360$^{\circ}$ Videos
The recent development of commodity 360$^{\circ}$ cameras have enabled a single video to capture an entire scene, which endows promising potentials in surveillance scenarios. However, research in omnidirectional video analysis has lagged behind the hardware advances. In this work, we address the important problem of action recognition in top-view 360$^{\circ}$ videos. Due to the wide filed-of-view, 360$^{\circ}$ videos usually capture multiple people performing actions at the same time. Furthermore, the appearance of people are deformed. The proposed framework first transforms omnidirectional videos into panoramic videos, then it extracts spatial-temporal features using region-based 3D CNNs for action recognition. We propose a weakly-supervised method based on multi-instance multi-label learning, which trains the model to recognize and localize multiple actions in a video using only video-level action labels as supervision. We perform experiments to quantitatively validate the efficacy of the proposed method and qualitatively demonstrate action localization results. To enable research in this direction, we introduce 360Action, the first omnidirectional video dataset for multi-person action recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationAction RecognitionMulti-Label LearningSimilar Papers 제목 키워드 기반
Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos
Despite the recent advances in video classification, progress in spatio-temporal action recognition has lagged behind. A major contributing factor has been the prohibitive cost of annotating videos frame-by-frame. In thi…
Action DetectionAction RecognitionMultiple Instance LearningSpatio-temporal Action Recognition+1Learning from Video and Text via Large-Scale Discriminative Clustering
Discriminative clustering has been successfully applied to a number of weakly-supervised learning tasks. Such applications include person and action recognition, text-to-video alignment, object co-segmentation and coloca…
Action RecognitionClusteringTemporal Action LocalizationVideo Alignment+3Re-ID-AR: Improved Person Re-identification in Video via Joint Weakly Supervised Action Recognition
We uniquely consider the task of joint person re-identification (Re-ID) and action recognition in video as a multi-task problem. In addition to the broader potential of joint Re-ID and action recognition within the conte…
Action RecognitionPerson Re-IdentificationVideo UnderstandingWeakly-Supervised Action RecognitionWeakly-supervised Multi-task Learning for Multimodal Affect Recognition
Multimodal affect recognition constitutes an important aspect for enhancing interpersonal relationships in human-computer interaction. However, relevant data is hard to come by and notably costly to annotate, which poses…
Emotion RecognitionMulti-Task LearningSentiment AnalysisWeakly supervised discriminative feature learning with state information for person identification
Unsupervised learning of identity-discriminative visual feature is appealing in real-world tasks where manual labelling is costly. However, the images of an identity can be visually discrepant when images are taken under…
Face RecognitionPerson IdentificationPerson Re-IdentificationPseudo Label+2