Transformer-based Global 3D Hand Pose Estimation in Two Hands Manipulating Objects Scenarios
This report describes our 1st place solution to ECCV 2022 challenge on Human Body, Hands, and Activities (HBHA) from Egocentric and Multi-view Cameras (hand pose estimation). In this challenge, we aim to estimate global 3D hand poses from the input image where two hands and an object are interacting on the egocentric viewpoint. Our proposed method performs end-to-end multi-hand pose estimation via transformer architecture. In particular, our method robustly estimates hand poses in a scenario where two hands interact. Additionally, we propose an algorithm that considers hand scales to robustly estimate the absolute depth. The proposed algorithm works well even when the hand sizes are various for each person. Our method attains 14.4 mm (left) and 15.9 mm (right) errors for each hand in the test set.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Hand Pose EstimationHand Pose EstimationPose EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image
3D interacting hand pose estimation from a single RGB image is a challenging task, due to serious self-occlusion and inter-occlusion towards hands, confusing similar appearance patterns between 2 hands, ill-posed joint p…
3D Interacting Hand Pose EstimationHand Pose EstimationPose EstimationPose PredictionTwo-hand Global 3D Pose Estimation Using Monocular RGB
We tackle the challenging task of estimating global 3D joint locations for both hands via only monocular RGB input images. We propose a novel multi-stage convolutional neural network based pipeline that accurately segmen…
3D Canonical Hand Pose Estimation3D Hand Pose Estimation3D Pose EstimationHand Pose Estimation+2Ego2HandsPose: A Dataset for Egocentric Two-hand 3D Global Pose Estimation
Color-based two-hand 3D pose estimation in the global coordinate system is essential in many applications. However, there are very few datasets dedicated to this task and no existing dataset supports estimation in a non-…
3D Pose EstimationDiversityHand SegmentationPose Estimation+1Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
Since the emergence of large annotated datasets, state-of-the-art hand pose estimation methods have been mostly based on discriminative learning. Recently, a hybrid approach has embedded a kinematic layer into the deep l…
Hand Pose EstimationPose EstimationHandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network
Hands are often severely occluded by objects, which makes 3D hand mesh estimation challenging. Previous works often have disregarded information at occluded regions. However, we argue that occluded regions have strong co…
3D Hand Pose Estimationhand-object pose