3D Pose from Motion for Cross-view Action Recognition via Non-linear Circulant Temporal Encoding
We describe a new approach to transfer knowledge across views for action recognition by using examples from a large collection of unlabelled mocap data. We achieve this by directly matching purely motion based features from videos to mocap. Our approach recovers 3D pose sequences without performing any body part tracking. We use these matches to generate multiple motion projections and thus add view invariance to our action recognition model. We also introduce a closed form solution for approximate non-linear Circulant Temporal Encoding (nCTE), which allows us to efficiently perform the matches in the frequency domain. We test our approach on the challenging unsupervised modality of the IXMAS dataset, and use publicly available motion capture data for matching. Without any additional annotation effort, we are able to significantly outperform the current state of the art.
Code (1)
Tasks
Action RecognitionTemporal Action LocalizationSimilar Papers 제목 키워드 기반
MAVR-Net: Robust Multi-View Learning for MAV Action Recognition with Cross-View Attention
Recognizing the motion of Micro Aerial Vehicles (MAVs) is crucial for enabling cooperative perception and control in autonomous aerial swarms. Yet, vision-based recognition models relying only on RGB data often fail to c…
Action RecognitionCross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions
Robustness to domain changes is a key capability for effective deployment of human action recognition systems in real-world scenarios, where action categories at inference can present important domain shifts or even unse…
Zero-Shot Action RecognitionTransfer LearningCross-view Action Modeling, Learning and Recognition
Existing methods on video-based action recognition are generally view-dependent, i.e., performing recognition from the same views seen in the training data. We present a novel multiview spatio-temporal AND-OR graph (MST-…
Action RecognitionTemporal Action LocalizationView-invariant Deep Architecture for Human Action Recognition using late fusion
Human action Recognition for unknown views is a challenging task. We propose a view-invariant deep human action recognition framework, which is a novel integration of two important action cues: motion and shape temporal …
Action RecognitionSSIMTemporal Action LocalizationUnsupervised Learning of View-invariant Action Representations
The recent success in human action recognition with deep learning methods mostly adopt the supervised learning paradigm, which requires significant amount of manually labeled data to achieve good performance. However, la…
Action RecognitionRepresentation LearningTemporal Action Localization