Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling
Most of the previous 3D human pose estimation work relied on the powerful memory capability of the network to obtain suitable 2D-3D mappings from the training data. Few works have studied the modeling of human posture deformation in motion. In this paper, we propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton, and a frame-by-frame skeleton deformation. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence, and then sum them to obtain the pose of each frame. Subsequently, a loss term based on the diffusion model is used to ensure that the pipeline learns the correct prior motion knowledge. Finally, we have evaluated our proposed method on mainstream datasets and obtained superior results outperforming the state-of-the-art.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Human Pose Estimation3D Pose EstimationPose EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
EffiScene: Efficient Per-Pixel Rigidity Inference for Unsupervised Joint Learning of Optical Flow, Depth, Camera Pose and Motion Segmentation
This paper addresses the challenging unsupervised scene flow estimation problem by jointly learning four low-level vision sub-tasks: optical flow $\textbf{F}$, stereo-depth $\textbf{D}$, camera pose $\textbf{P}$ and moti…
Depth EstimationMotion SegmentationOptical Flow EstimationScene Flow Estimation+1Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry
In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By an…
3D geometryAutonomous DrivingDepth And Camera MotionDepth Estimation+3Unsupervised Monocular Depth Reconstruction of Non-Rigid Scenes
Monocular depth reconstruction of complex and dynamic scenes is a highly challenging problem. While for rigid scenes learning-based methods have been offering promising results even in unsupervised cases, there exists li…
Depth EstimationMotion SegmentationUnOS: Unified Unsupervised Optical-Flow and Stereo-Depth Estimation by Watching Videos
In this paper, we propose UnOS, an unified system for unsupervised optical flow and stereo depth estimation using convolutional neural network (CNN) by taking advantages of their inherent geometrical consistency based on…
Depth EstimationMotion SegmentationOptical Flow EstimationStereo Depth Estimation+1Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation
A truly generalizable approach to rigid segmentation and motion estimation is fundamental to 3D understanding of articulated objects and moving scenes. In view of the closely intertwined relationship between segmentation…
Computational EfficiencyMotion EstimationSegmentation