HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation
Hand-object pose estimation (HOPE) aims to jointly detect the poses of both a hand and of a held object. In this paper, we propose a lightweight model called HOPE-Net which jointly estimates hand and object pose in 2D and 3D in real-time. Our network uses a cascade of two adaptive graph convolutional neural networks, one to estimate 2D coordinates of the hand joints and object corners, followed by another to convert 2D coordinates to 3D. Our experiments show that through end-to-end training of the full network, we achieve better accuracy for both the 2D and 3D coordinate estimation problems. The proposed 2D to 3D graph convolution-based model could be applied to other 3D landmark detection problems, where it is possible to first predict the 2D keypoints and then transform them to 3D.
Code (1)
Tasks
hand-object poseObjectPose EstimationSimilar Papers 제목 키워드 기반
UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation
Estimating the 3D pose of hand and potential hand-held object from monocular images is a longstanding challenge. Yet, existing methods are specialized, focusing on either bare-hand or hand interacting with object. No…
hand-object poseHand Pose EstimationObjectPose EstimationHOPE: Hand-Object Pressure Estimation from Monocular Videos
Estimating physical pressure from vision is essential for understanding contact-rich hand-object interaction. However, prior vision-based pressure estimation methods are largely limited to planar surfaces and single imag…
Video PredictionViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion
In this letter, we introduce ViHOPE, a novel framework for estimating the 6D pose of an in-hand object using visuotactile perception. Our key insight is that the accuracy of the 6D object pose estimate can be improved by…
6D Pose EstimationGenerative Adversarial NetworkObjectPose EstimationPOV-Surgery: A Dataset for Egocentric Hand and Tool Pose Estimation During Surgical Activities
The surgical usage of Mixed Reality (MR) has received growing attention in areas such as surgical navigation systems, skill assessment, and robot-assisted surgeries. For such applications, pose estimation for hand and su…
3D Hand Pose Estimationhand-object poseMixed RealityPose Estimation+2Object category understanding via eye fixations on freehand sketches
The study of eye gaze fixations on photographic images is an active research area. In contrast, the image subcategory of freehand sketches has not received as much attention for such studies. In this paper, we analyze th…
Object