Skeleton Boxes: Solving skeleton based action detection with a single deep convolutional neural network
Action recognition from well-segmented 3D skeleton video has been intensively studied. However, due to the difficulty in representing the 3D skeleton video and the lack of training data, action detection from streaming 3D skeleton video still lags far behind its recognition counterpart and image based object detection. In this paper, we propose a novel approach for this problem, which leverages both effective skeleton video encoding and deep regression based object detection from images. Our framework consists of two parts: skeleton-based video image mapping, which encodes a skeleton video to a color image in a temporal preserving way, and an end-to-end trainable fast skeleton action detector (Skeleton Boxes) based on image detection. Experimental results on the latest and largest PKU-MMD benchmark dataset demonstrate that our method outperforms the state-of-the-art methods with a large margin. We believe our idea would inspire and benefit future research in this important area.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionAction Recognitionobject-detectionObject DetectionTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Skeleton-based Approaches based on Machine Vision: A Survey
Recently, skeleton-based approaches have achieved rapid progress on the basis of great success in skeleton representation. Plenty of researches focus on solving specific problems according to skeleton features. Some skel…
object-detectionObject DetectionSurveyDeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images
Object skeletons are useful for object representation and object detection. They are complementary to the object contour, and provide extra information, such as how object scale (thickness) varies among object parts. But…
Multi-Task LearningObjectobject-detectionObject Detection+1Skeleton-OOD: An End-to-End Skeleton-Based Model for Robust Out-of-Distribution Human Action Detection
Human action recognition is crucial in computer vision systems. However, in real-world scenarios, human actions often fall outside the distribution of training data, requiring a model to both recognize in-distribution (I…
Action DetectionAction RecognitionSkeleton Based Action RecognitionTemporal Action LocalizationSkeleton-based Action Recognition with Convolutional Neural Networks
Current state-of-the-art approaches to skeleton-based action recognition are mostly based on recurrent neural networks (RNN). In this paper, we propose a novel convolutional neural networks (CNN) based framework for both…
Action ClassificationAction DetectionAction RecognitionGeneral Classification+2SkeleTR: Towards Skeleton-based Action Recognition in the Wild
We present SkeleTR, a new framework for skeleton-based action recognition. In contrast to prior work, which focuses mainly on controlled environments, we target in-the-wild scenarios that typically involve a variable…
Action ClassificationAction DetectionAction RecognitionActivity Recognition+3