Learning Human Optical Flow
The optical flow of humans is well known to be useful for the analysis of human action. Given this, we devise an optical flow algorithm specifically for human motion and show that it is superior to generic flow methods. Designing a method by hand is impractical, so we develop a new training database of image sequences with ground truth optical flow. For this we use a 3D model of the human body and motion capture data to synthesize realistic flow fields. We then train a convolutional neural network to estimate human flow fields from pairs of images. Since many applications in human motion analysis depend on speed, and we anticipate mobile applications, we base our method on SpyNet with several modifications. We demonstrate that our trained network is more accurate than a wide range of top methods on held-out test data and that it generalizes well to real image sequences. When combined with a person detector/tracker, the approach provides a full solution to the problem of 2D human flow estimation. Both the code and the dataset are available for research.
Code (1)
Tasks
Optical Flow EstimationSimilar Papers 제목 키워드 기반
Bootstrapping Human Optical Flow and Pose
We propose a bootstrapping framework to enhance human optical flow and pose. We show that, for videos involving humans in scenes, we can improve both the optical flow and the pose estimation quality of humans by consider…
Optical Flow EstimationPose EstimationLearning Multi-Human Optical Flow
The optical flow of humans is well known to be useful for the analysis of human action. Recent optical flow methods focus on training deep networks to approach the problem. However, the training data used by them does no…
Optical Flow EstimationCombining Spatio-Temporal Appearance Descriptors and Optical Flow for Human Action Recognition in Video Data
This paper proposes combining spatio-temporal appearance (STA) descriptors with optical flow for human action recognition. The STA descriptors are local histogram-based descriptors of space-time, suitable for building a …
Action RecognitionOptical Flow EstimationTemporal Action LocalizationHuPerFlow: A Comprehensive Benchmark for Human vs. Machine Motion Estimation Comparison
As AI models are increasingly integrated into applications involving human interaction, understanding the alignment between human perception and machine vision has become essential. One example is the estimation of v…
Motion EstimationOptical Flow EstimationOmniFlow: Human Omnidirectional Optical Flow
Optical flow is the motion of a pixel between at least two consecutive video frames and can be estimated through an end-to-end trainable convolutional neural network. To this end, large training datasets are required to …
Optical Flow Estimation