Event-based Timestamp Image Encoding Network for Human Action Recognition and Anticipation
Event camera is an asynchronous, high frequency vision sensor with low power consumption, which is suitable for human action understanding task. It is vital to encode the spatial-temporal information of event data properly and use standard computer vision tool to learn from the data. In this work, we propose a timestamp image encoding 2D network, which takes the encoded spatial-temporal images with polarity information of the event data as input and output the action label. In addition, we propose a future timestamp image generator to generate futureaction information to aid the model to anticipate the human action when the action is not completed. Experiment results show that our method can achieve the same level of performance as those RGB-based benchmarks on real world action recognition,and also achieve the state of the art (SOTA) result on gesture recognition. Our future timestamp image generating model can effectively improve the prediction accuracy when the action is not completed. We also provide insight discussion on the importance of motion and appearance information in action recognition and anticipation.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionAction UnderstandingGesture RecognitionTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Event-based Action Recognition Using Timestamp Image Encoding Network
Event camera is an asynchronous, high frequency vision sensor with low power consumption, which is suitable for human action recognition task. It is vital to encode the spatial-temporal information of event data properly…
Action RecognitionGesture RecognitionTemporal Action LocalizationHuman Activity Recognition: A Spatio-temporal Image Encoding of 3D Skeleton Data for Online Action Detection
Human activity recognition (HAR) based on skeleton data that can be extracted from videos (Kinect for example) , or provided by a depth camera is a time series classification problem, where handling both spatial and temp…
Action DetectionActivity RecognitionHuman Activity RecognitionOnline Action Detection+2Lossy Event Compression based on Image-derived Quad Trees and Poisson Disk Sampling
With several advantages over conventional RGB cameras, event cameras have provided new opportunities for tackling visual tasks under challenging scenarios with fast motion, high dynamic range, and/or power constraint. Ye…
Event-based visionImage ReconstructionVideo CompressionExplaining Categorical Feature Interactions Using Graph Covariance and LLMs
Modern datasets often consist of numerous samples with abundant features and associated timestamps. Analyzing such datasets to uncover underlying events typically requires complex statistical methods and substantial doma…
BinarizationMarked Temporal Bayesian Flow Point Processes
Marked event data captures events by recording their continuous-valued occurrence timestamps along with their corresponding discrete-valued types. They have appeared in various real-world scenarios such as social media, …
Point Processes