Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment
Video sequences exhibit significant nuisance variations (undesired effects) of speed of actions, temporal locations, and subjects' poses, leading to temporal-viewpoint misalignment when comparing two sets of frames or evaluating the similarity of two sequences. Thus, we propose Joint tEmporal and cAmera viewpoiNt alIgnmEnt (JEANIE) for sequence pairs. In particular, we focus on 3D skeleton sequences whose camera and subjects' poses can be easily manipulated in 3D. We evaluate JEANIE on skeletal Few-shot Action Recognition (FSAR), where matching well temporal blocks (temporal chunks that make up a sequence) of support-query sequence pairs (by factoring out nuisance variations) is essential due to limited samples of novel classes. Given a query sequence, we create its several views by simulating several camera locations. For a support sequence, we match it with view-simulated query sequences, as in the popular Dynamic Time Warping (DTW). Specifically, each support temporal block can be matched to the query temporal block with the same or adjacent (next) temporal index, and adjacent camera views to achieve joint local temporal-viewpoint warping. JEANIE selects the smallest distance among matching paths with different temporal-viewpoint warping patterns, an advantage over DTW which only performs temporal alignment. We also propose an unsupervised FSAR akin to clustering of sequences with JEANIE as a distance measure. JEANIE achieves state-of-the-art results on NTU-60, NTU-120, Kinetics-skeleton and UWA3D Multiview Activity II on supervised and unsupervised FSAR, and their meta-learning inspired fusion.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionDynamic Time WarpingFew-Shot action recognitionFew Shot Action RecognitionMeta-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
3D Skeleton-based Few-shot Action Recognition with JEANIE is not so Naïve
In this paper, we propose a Few-shot Learning pipeline for 3D skeleton-based action recognition by Joint tEmporal and cAmera viewpoiNt alIgnmEnt (JEANIE). To factor out misalignment between query and support sequences of…
Action RecognitionDynamic Time WarpingFew-Shot action recognitionFew Shot Action Recognition+3Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition
We propose a Few-shot Learning pipeline for 3D skeleton-based action recognition by Joint tEmporal and cAmera viewpoiNt alIgnmEnt (JEANIE). To factor out misalignment between query and support sequences of 3D body joints…
Action RecognitionDynamic Time WarpingFew-Shot action recognitionFew Shot Action Recognition+3Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning
Skeleton-based action representation learning aims to interpret and understand human behaviors by encoding the skeleton sequences, which can be categorized into two primary training paradigms: supervised learning and sel…
Action RecognitionContrastive LearningKnowledge DistillationRepresentation Learning+4SimMC: Simple Masked Contrastive Learning of Skeleton Representations for Unsupervised Person Re-Identification
Recent advances in skeleton-based person re-identification (re-ID) obtain impressive performance via either hand-crafted skeleton descriptors or skeleton representation learning with deep learning paradigms. However, the…
Contrastive LearningPerson Re-IdentificationRepresentation LearningUnsupervised Person Re-IdentificationParallel Attention Interaction Network for Few-Shot Skeleton-Based Action Recognition
Learning discriminative features from very few labeled samples to identify novel classes has received increasing attention in skeleton-based action recognition. Existing works aim to learn action-specific embeddings …
Action RecognitionFew-Shot Skeleton-Based Action RecognitionSkeleton Based Action Recognition