User-Specific Hand Modeling from Monocular Depth Sequences
This paper presents a method for acquiring dense nonrigid shape and deformation from a single monocular depth sensor. We focus on modeling the human hand, and assume that a single rough template model is available. We combine and extend existing work on model-based tracking, subdivision surface fitting, and mesh deformation to acquire detailed hand models from as few as 15 frames of depth data. We propose an objective that measures the error of fit between each sampled data point and a continuous model surface defined by a rigged control mesh, and uses as-rigid-as-possible (ARAP) regularizers to cleanly separate the model and template geometries. A key contribution is our use of a smooth model based on subdivision surfaces that allows simultaneous optimization over both correspondences and model parameters. This avoids the use of iterated closest point (ICP) algorithms which often lead to slow convergence. Automatic initialization is obtained using a regression forest trained to infer approximate correspondences. Experiments show that the resulting meshes model the user's hand shape more accurately than just adapting the shape parameters of the skeleton, and that the retargeted skeleton accurately models the user's articulations. We investigate the effect of various modeling choices, and show the benefits of using subdivision surfaces and ARAP regularization.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Focusable Monocular Depth Estimation
Monocular depth foundation models generalize well across scenes, yet they are typically optimized with uniform pixel-wise objectives that do not distinguish user-specified or task-relevant target regions from the surroun…
Monocular Depth EstimationEnhanced Scale-aware Depth Estimation for Monocular Endoscopic Scenes with Geometric Modeling
Scale-aware monocular depth estimation poses a significant challenge in computer-aided endoscopic navigation. However, existing depth estimation methods that do not consider the geometric priors struggle to learn the abs…
Depth EstimationMonocular Depth EstimationEgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulatio…
Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection
As a crucial task of autonomous driving, 3D object detection has made great progress in recent years. However, monocular 3D object detection remains a challenging problem due to the unsatisfactory performance in depth es…
3D Object DetectionAutonomous DrivingDepth EstimationMonocular 3D Object Detection+4Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction
Classical monocular Simultaneous Localization And Mapping (SLAM) and the recently emerging convolutional neural networks (CNNs) for monocular depth prediction represent two largely disjoint approaches towards building a …
Depth EstimationDepth PredictionSimultaneous Localization and Mapping