On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation
Supervised approaches to 3D pose estimation from single images are remarkably effective when labeled data is abundant. However, as the acquisition of ground-truth 3D labels is labor intensive and time consuming, recent attention has shifted towards semi- and weakly-supervised learning. Generating an effective form of supervision with little annotations still poses major challenge in crowded scenes. In this paper we propose to impose multi-view geometrical constraints by means of a weighted differentiable triangulation and use it as a form of self-supervision when no labels are available. We therefore train a 2D pose estimator in such a way that its predictions correspond to the re-projection of the triangulated 3D pose and train an auxiliary network on them to produce the final 3D poses. We complement the triangulation with a weighting mechanism that alleviates the impact of noisy predictions caused by self-occlusion or occlusion from other subjects. We demonstrate the effectiveness of our semi-supervised approach on Human3.6M and MPI-INF-3DHP datasets, as well as on a new multi-view multi-person dataset that features occlusion.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Human Pose Estimation3D Pose EstimationFormPose EstimationWeakly-supervised 3D Human Pose EstimationWeakly-supervised LearningSimilar Papers 제목 키워드 기반
Triangulation Residual Loss for Data-efficient 3D Pose Estimation
This paper presents Triangulation Residual loss (TR loss) for multiview 3D pose estimation in a data-efficient manner. Existing 3D supervised models usually require large-scale 3D annotated datasets, but the amount of ex…
Supervision by Registration and Triangulation for Landmark Detection
We present Supervision by Registration and Triangulation (SRT), an unsupervised approach that utilizes unlabeled multi-view video to improve the accuracy and precision of landmark detectors. Being able to utilize unlabel…
Optical Flow EstimationMEOM: Multi-View Expected-OKS Maximization for Human Pose Triangulation
Conventional algebraic triangulation solves 3D human pose estimation (HPE) from multi-view 2D keypoints. The typical approach, decoding 2D keypoints from predicted heatmaps, is unreliable as heatmaps can be multimodal un…
3D Human Pose EstimationRevisit Self-supervised Depth Estimation with Local Structure-from-Motion
Both self-supervised depth estimation and Structure-from-Motion (SfM) recover scene depth from RGB videos. Despite sharing a similar objective, the two approaches are disconnected. Prior works of self-supervision backpro…
Depth EstimationNeRFLearning Optical Flow, Depth, and Scene Flow without Real-World Labels
Self-supervised monocular depth estimation enables robots to learn 3D perception from raw video streams. This scalable approach leverages projective geometry and ego-motion to learn via view synthesis, assuming the world…
Autonomous DrivingDepth EstimationMonocular Depth EstimationMulti-Task Learning+2