Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning
We propose a method for hand pose estimation based on a deep regressor trained on two different kinds of input. Raw depth data is fused with an intermediate representation in the form of a segmentation of the hand into parts. This intermediate representation contains important topological information and provides useful cues for reasoning about joint locations. The mapping from raw depth to segmentation maps is learned in a semi/weakly-supervised way from two different datasets: (i) a synthetic dataset created through a rendering pipeline including densely labeled ground truth (pixelwise segmentations); and (ii) a dataset with real images for which ground truth joint positions are available, but not dense segmentations. Loss for training on real images is generated from a patch-wise restoration process, which aligns tentative segmentation maps with a large dictionary of synthetic poses. The underlying premise is that the domain shift between synthetic and real data is smaller in the intermediate representation, where labels carry geometric and topological meaning, than in the raw input domain. Experiments on the NYU dataset show that the proposed training method decreases error on joints over direct regression of joints from depth data by 15.7%.
Code (0)
등록된 구현이 없습니다.
Tasks
Hand Pose EstimationPose EstimationSegmentationWeakly-supervised LearningSimilar Papers 제목 키워드 기반
SemiHand: Semi-Supervised Hand Pose Estimation With Consistency
We present SemiHand, a semi-supervised framework for 3D hand pose estimation from monocular images. We pre-train the model on labelled synthetic data and fine-tune it on unlabelled real-world data by pseudo-labeling …
3D Hand Pose EstimationData AugmentationHand Pose EstimationPose EstimationSO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation With Semi-Supervised Learning
3D hand pose estimation has made significant progress recently, where Convolutional Neural Networks (CNNs) play a critical role. However, most of the existing CNN-based hand pose estimation methods depend much on the tra…
3D Hand Pose EstimationDecoderHand Pose EstimationPose EstimationSemi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time
Estimating 3D hand and object pose from a single image is an extremely challenging problem: hands and objects are often self-occluded during interactions, and the 3D annotations are scarce as even humans cannot directly …
3D Hand Pose Estimationhand-object poseHand Pose EstimationObject+1S$^2$Contact: Graph-based Network for 3D Hand-Object Contact Estimation with Semi-Supervised Learning
Despite the recent efforts in accurate 3D annotations in hand and object datasets, there still exist gaps in 3D hand and object reconstructions. Existing works leverage contact maps to refine inaccurate hand-object pose …
hand-object poseObjectSemi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learning
3D hand-object pose estimation is an important issue to understand the interaction between human and environment. Current hand-object pose estimation methods require detailed 3D labels, which are expensive and labor-inte…
Dictionary Learninghand-object poseObjectPose Estimation