HMP: Hand Motion Priors for Pose and Shape Estimation from Video
Understanding how humans interact with the world necessitates accurate 3D hand pose estimation, a task complicated by the hand's high degree of articulation, frequent occlusions, self-occlusions, and rapid motions. While most existing methods rely on single-image inputs, videos have useful cues to address aforementioned issues. However, existing video-based 3D hand datasets are insufficient for training feedforward models to generalize to in-the-wild scenarios. On the other hand, we have access to large human motion capture datasets which also include hand motions, e.g. AMASS. Therefore, we develop a generative motion prior specific for hands, trained on the AMASS dataset which features diverse and high-quality hand motions. This motion prior is then employed for video-based 3D hand motion estimation following a latent optimization approach. Our integration of a robust motion prior significantly enhances performance, especially in occluded scenarios. It produces stable, temporally consistent results that surpass conventional single-frame methods. We demonstrate our method's efficacy via qualitative and quantitative evaluations on the HO3D and DexYCB datasets, with special emphasis on an occlusion-focused subset of HO3D. Code is available at https://hmp.is.tue.mpg.de
Code (0)
등록된 구현이 없습니다.
Tasks
3D Hand Pose EstimationHand Pose EstimationMotion EstimationPose EstimationSimilar Papers 제목 키워드 기반
SAMP: Shape and Motion Priors for 4D Vehicle Reconstruction
Inferring the pose and shape of vehicles in 3D from a movable platform still remains a challenging task due to the projective sensing principle of cameras, difficult surface properties e.g. reflections or transparency, a…
Pose EstimationHand-held Object Reconstruction from RGB Video with Dynamic Interaction
This work aims to reconstruct the 3D geometry of a rigid object manipulated by one or both hands using monocular RGB video. Previous methods rely on Structure-from-Motion or hand priors to estimate relative motion be…
3D Generation3D geometryObjectObject Reconstruction+14DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface
Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics. However, due to occlusions and physical limitations of range sensors, existing methods o…
Motion EstimationTowards Accurate Markerless Human Shape and Pose Estimation over Time
Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios. Here we propose a fully automatic method that give…
Pose EstimationShapeGaussian: High-Fidelity 4D Human Reconstruction in Monocular Videos via Vision Priors
We introduce ShapeGaussian, a high-fidelity, template-free method for 4D human reconstruction from casual monocular videos. Generic reconstruction methods lacking robust vision priors, such as 4DGS, struggle to capture h…
Pose Estimation