InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

2020-08-21 · ECCV 2020 8 · Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, Kyoung Mu Lee

Analysis of hand-hand interactions is a crucial step towards better understanding human behavior. However, most researches in 3D hand pose estimation have focused on the isolated single hand case. Therefore, we firstly propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network, InterNet, for 3D interacting hand pose estimation from a single RGB image. The proposed InterHand2.6M consists of \textbf{2.6M labeled single and interacting hand frames} under various poses from multiple subjects. Our InterNet simultaneously performs 3D single and interacting hand pose estimation. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset. Finally, we show 3D interacting hand pose estimation results from general images. Our code and dataset are available at https://mks0601.github.io/InterHand2.6M/.

📄 PDF Abstract BibTeX arXiv:2008.09309

Code (2)

facebookresearch/InterHand2.6M pytorch

open-mmlab/mmpose pytorch

Tasks

3D Hand Pose Estimation 3D Interacting Hand Pose Estimation Hand Pose Estimation Pose Estimation

A Dataset of Relighted 3D Interacting Hands

2023-10-26 · NeurIPS 2023 11

The two-hand interaction is one of the most challenging signals to analyze due to the self-similarity, complicated articulations, and occlusions of hands. Although several datasets have been proposed for the two-hand int…

3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal

2022-07-22 · Hao Meng, Sheng Jin, Wentao Liu, Chen Qian 외

Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. Unlike most previous works that directly predict the 3D poses of two interacting hands simultaneously, we propose …

3D Interacting Hand Pose Estimation Hand Pose Estimation Pose Estimation

HandNeRF: Neural Radiance Fields for Animatable Interacting Hands

2023-03-24 · CVPR 2023 1 · Zhiyang Guo, Wengang Zhou, Min Wang, Li Li 외

We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands, enabling the rendering of photo-realistic images and videos for gesture animation fro…

NeRF

ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

2023-03-10 · CVPR 2023 1 · Zhengdi Yu, Shaoli Huang, Chen Fang, Toby P. Breckon 외

Reconstructing two hands from monocular RGB images is challenging due to frequent occlusion and mutual confusion. Existing methods mainly learn an entangled representation to encode two interacting hands, which are incre…

3D Interacting Hand Pose Estimation 3D Reconstruction Vocal Bursts Valence Prediction

Interacting Attention Graph for Single Image Two-Hand Reconstruction

2022-03-17 · CVPR 2022 1 · Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu 외

Graph convolutional network (GCN) has achieved great success in single hand reconstruction task, while interacting two-hand reconstruction by GCN remains unexplored. In this paper, we present Interacting Attention Graph …

3D Interacting Hand Pose Estimation Vocal Bursts Valence Prediction