MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation
Cross view feature fusion is the key to address the occlusion problem in human pose estimation. The current fusion methods need to train a separate model for every pair of cameras making them difficult to scale. In this work, we introduce MetaFuse, a pre-trained fusion model learned from a large number of cameras in the Panoptic dataset. The model can be efficiently adapted or finetuned for a new pair of cameras using a small number of labeled images. The strong adaptation power of MetaFuse is due in large part to the proposed factorization of the original fusion model into two parts (1) a generic fusion model shared by all cameras, and (2) lightweight camera-dependent transformations. Furthermore, the generic model is learned from many cameras by a meta-learning style algorithm to maximize its adaptation capability to various camera poses. We observe in experiments that MetaFuse finetuned on the public datasets outperforms the state-of-the-arts by a large margin which validates its value in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningPose EstimationSimilar Papers 제목 키워드 기반
COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation
Estimating global human motion from moving cameras is challenging due to the entanglement of human and camera motions. To mitigate the ambiguity, existing methods leverage learned human motion priors, which however often…
Motion EstimationAdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild
Occlusion is probably the biggest challenge for human pose estimation in the wild. Typical solutions often rely on intrusive sensors such as IMUs to detect occluded joints. To make the task truly unconstrained, we presen…
3D Human Pose EstimationPose EstimationUnsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Sampling
We propose a method of estimating a 3D human pose from a single view without 3D supervision. The key to our method is to leverage the 2D diffusion priors of motion diffusion models (MDMs) pre-trained on large 2D human po…
Unsupervised 3D Human Pose Estimation3D Pose EstimationLite Pose: Efficient Architecture Design for 2D Human Pose Estimation
Pose estimation plays a critical role in human-centered vision applications. However, it is difficult to deploy state-of-the-art HRNet-based pose estimation models on resource-constrained edge devices due to the high com…
2D Human Pose EstimationMulti-Person Pose EstimationPose EstimationSDPose: Exploiting Diffusion Priors for Out-of-Domain and Robust Pose Estimation
Pre-trained diffusion models provide rich latent features across U-Net levels and are emerging as powerful vision backbones. While prior works such as Marigold and Lotus repurpose diffusion priors for dense geometric per…
Domain GeneralizationPose Estimation