Unsupervised 3D Keypoint Discovery with Multi-View Geometry
Analyzing and training 3D body posture models depend heavily on the availability of joint labels that are commonly acquired through laborious manual annotation of body joints or via marker-based joint localization using carefully curated markers and capturing systems. However, such annotations are not always available, especially for people performing unusual activities. In this paper, we propose an algorithm that learns to discover 3D keypoints on human bodies from multiple-view images without any supervision or labels other than the constraints multiple-view geometry provides. To ensure that the discovered 3D keypoints are meaningful, they are re-projected to each view to estimate the person's mask that the model itself has initially estimated without supervision. Our approach discovers more interpretable and accurate 3D keypoints compared to other state-of-the-art unsupervised approaches on Human3.6M and MPI-INF-3DHP benchmark datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Human Pose EstimationKeypoint EstimationPose EstimationUnsupervised 3D Human Pose EstimationSimilar Papers 제목 키워드 기반
Weakly Supervised Keypoint Discovery
In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision. Recent works on unsupervised keypoint discovery reliably discover keypoints of aligned instances. However, when the…
Conditional Image GenerationImage GenerationKeypoint EstimationWeakly-supervised LearningUnsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priors
This paper introduces KeyDiff3D, a framework for unsupervised monocular 3D keypoints estimation that accurately predicts 3D keypoints from a single image. While previous methods rely on manual annotations or calibrated m…
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
Quantifying motion in 3D is important for studying the behavior of humans and other animals, but manual pose annotations are expensive and time-consuming to obtain. Self-supervised keypoint discovery is a promising strat…
DecoderUnsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects. Our method learns the 3D structure and dynamics of objects solely from single-view RGB videos, and can decompose them into semant…
3D geometryKeypoint EstimationNovel View SynthesisUnsupervised Depth Estimation, 3D Face Rotation and Replacement
We present an unsupervised approach for learning to estimate three dimensional (3D) facial structure from a single image while also predicting 3D viewpoint transformations that match a desired pose and facial geometry. W…
Depth EstimationTranslation