Appearance Consensus Driven Self-Supervised Human Mesh Recovery
We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images in the absence of any paired supervision. Recent advances have shifted the interest towards directly regressing parameters of a parametric human model by supervising them on large-scale datasets with 2D landmark annotations. This limits the generalizability of such approaches to operate on images from unlabeled wild environments. Acknowledging this we propose a novel appearance consensus driven self-supervised objective. To effectively disentangle the foreground (FG) human we rely on image pairs depicting the same person (consistent FG) in varied pose and background (BG) which are obtained from unlabeled wild videos. The proposed FG appearance consistency objective makes use of a novel, differentiable Color-recovery module to obtain vertex colors without the need for any appearance network; via efficient realization of color-picking and reflectional symmetry. We achieve state-of-the-art results on the standard model-based 3D pose estimation benchmarks at comparable supervision levels. Furthermore, the resulting colored mesh prediction opens up the usage of our framework for a variety of appearance-related tasks beyond the pose and shape estimation, thus establishing our superior generalizability.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Pose EstimationHuman Mesh RecoveryPose EstimationSimilar Papers 제목 키워드 기반
Neural Reconstruction of Relightable Human Model from Monocular Video
Creating relightable and animatable human characters from monocular video at a low cost is a critical task for digital human modeling and virtual reality applications. This task is complex due to intricate articulati…
Human Detection and Segmentation via Multi-view Consensus
Self-supervised detection and segmentation of foreground objects aims for accuracy without annotated training data. However, existing approaches predominantly rely on restrictive assumptions on appearance and motion. For…
Human DetectionregressionSelf-Improving Code Generation via Semantic Entropy and Behavioral Consensus
Improving the code generation capabilities of large language models (LLMs) typically relies on supervised fine-tuning or preference optimization, both of which require costly external resources such as powerful teacher m…
Code GenerationOpineSum: Entailment-based self-training for abstractive opinion summarization
A typical product or place often has hundreds of reviews, and summarization of these texts is an important and challenging problem. Recent progress on abstractive summarization in domains such as news has been driven by …
Abstractive Text SummarizationArticlesFew-Shot LearningNatural Language Inference+1Understanding Pose and Appearance Disentanglement in 3D Human Pose Estimation
As 3D human pose estimation can now be achieved with very high accuracy in the supervised learning scenario, tackling the case where 3D pose annotations are not available has received increasing attention. In particular,…
3D Human Pose EstimationAdversarial AttackDisentanglementImage Generation+2