Analyzing the Synthetic-to-Real Domain Gap in 3D Hand Pose Estimation
Recent synthetic 3D human datasets for the face, body, and hands have pushed the limits on photorealism. Face recognition and body pose estimation have achieved state-of-the-art performance using synthetic training data alone, but for the hand, there is still a large synthetic-to-real gap. This paper presents the first systematic study of the synthetic-to-real gap of 3D hand pose estimation. We analyze the gap and identify key components such as the forearm, image frequency statistics, hand pose, and object occlusions. To facilitate our analysis, we propose a data synthesis pipeline to synthesize high-quality data. We demonstrate that synthetic hand data can achieve the same level of accuracy as real data when integrating our identified components, paving the path to use synthetic data alone for hand pose estimation. Code and data are available at: https://github.com/delaprada/HandSynthesis.git.
Code (1)
Tasks
3D Hand Pose EstimationFace RecognitionHand Pose EstimationPose EstimationSimilar Papers 제목 키워드 기반
Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles
Acquiring data to train deep learning-based object detectors on Unmanned Aerial Vehicles (UAVs) is expensive, time-consuming and may even be prohibited by law in specific environments. On the other hand, synthetic data i…
Objectobject-detectionObject DetectionH-GAN: the power of GANs in your Hands
We present HandGAN (H-GAN), a cycle-consistent adversarial learning approach implementing multi-scale perceptual discriminators. It is designed to translate synthetic images of hands to the real domain. Synthetic hands p…
Image-to-Image TranslationTone MappingTranslationAchieving Domain Robustness in Stereo Matching Networks by Removing Shortcut Learning
Learning-based stereo matching and depth estimation networks currently excel on public benchmarks with impressive results. However, state-of-the-art networks often fail to generalize from synthetic imagery to more challe…
Depth EstimationStereo MatchingSASMU: boost the performance of generalized recognition model using synthetic face dataset
Nowadays, deploying a robust face recognition product becomes easy with the development of face recognition techniques for decades. Not only profile image verification but also the state-of-the-art method can handle the …
Data AugmentationDomain GeneralizationFace RecognitionRobust Face RecognitionStRDAN: Synthetic-to-Real Domain Adaptation Network for Vehicle Re-Identification
Vehicle re-identification aims to obtain the same vehicles from vehicle images. This is challenging but essential for analyzing and predicting traffic flow in the city. Although deep learning methods have achieved enormo…
Domain AdaptationVehicle Re-Identification