Human Mesh Recovery from Arbitrary Multi-view Images
Human mesh recovery from arbitrary multi-view images involves two characteristics: the arbitrary camera poses and arbitrary number of camera views. Because of the variability, designing a unified framework to tackle this task is challenging. The challenges can be summarized as the dilemma of being able to simultaneously estimate arbitrary camera poses and recover human mesh from arbitrary multi-view images while maintaining flexibility. To solve this dilemma, we propose a divide and conquer framework for Unified Human Mesh Recovery (U-HMR) from arbitrary multi-view images. In particular, U-HMR consists of a decoupled structure and two main components: camera and body decoupling (CBD), camera pose estimation (CPE), and arbitrary view fusion (AVF). As camera poses and human body mesh are independent of each other, CBD splits the estimation of them into two sub-tasks for two individual sub-networks (ie, CPE and AVF) to handle respectively, thus the two sub-tasks are disentangled. In CPE, since each camera pose is unrelated to the others, we adopt a shared MLP to process all views in a parallel way. In AVF, in order to fuse multi-view information and make the fusion operation independent of the number of views, we introduce a transformer decoder with a SMPL parameters query token to extract cross-view features for mesh recovery. To demonstrate the efficacy and flexibility of the proposed framework and effect of each component, we conduct extensive experiments on three public datasets: Human3.6M, MPI-INF-3DHP, and TotalCapture.
Code (1)
Tasks
Camera Pose EstimationDecoderHuman Mesh RecoveryPose EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Monocular Models are Strong Learners for Multi-View Human Mesh Recovery
Multi-view human mesh recovery (HMR) is broadly deployed in diverse domains where high accuracy and strong generalization are essential. Existing approaches can be broadly grouped into geometry-based and learning-based m…
Human Mesh RecoverySkeleton2Mesh: Kinematics Prior Injected Unsupervised Human Mesh Recovery
In this paper, we decouple unsupervised human mesh recovery into the well-studied problems of unsupervised 3D pose estimation, and human mesh recovery from estimated 3D skeletons, focusing on the latter task. The cha…
3D Pose EstimationHuman Mesh RecoveryPose EstimationImplicit 3D Human Mesh Recovery using Consistency with Pose and Shape from Unseen-view
From an image of a person, we can easily infer the natural 3D pose and shape of the person even if ambiguity exists. This is because we have a mental model that allows us to imagine a person's appearance at different vie…
3D Human Pose EstimationHuman Mesh RecoverySelf-Supervised LearningDelving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery
Regression-based methods have shown high efficiency and effectiveness for multi-view human mesh recovery. The key components of a typical regressor lie in the feature extraction of input views and the fusion of multi-vie…
Human Mesh RecoveryMulti-view 3D Human Pose EstimationregressionMulti-view Human Body Mesh Translator
Existing methods for human mesh recovery mainly focus on single-view frameworks, but they often fail to produce accurate results due to the ill-posed setup. Considering the maturity of the multi-view motion capture syste…
Human Mesh Recovery