Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation
Monocular estimation of 3d human pose has attracted increased attention with the availability of large ground-truth motion capture datasets. However, the diversity of training data available is limited and it is not clear to what extent methods generalize outside the specific datasets they are trained on. In this work we carry out a systematic study of the diversity and biases present in specific datasets and its effect on cross-dataset generalization across a compendium of 5 pose datasets. We specifically focus on systematic differences in the distribution of camera viewpoints relative to a body-centered coordinate frame. Based on this observation, we propose an auxiliary task of predicting the camera viewpoint in addition to pose. We find that models trained to jointly predict viewpoint and pose systematically show significantly improved cross-dataset generalization.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Human Pose EstimationDiversityMonocular 3D Human Pose EstimationPose EstimationSimilar Papers 제목 키워드 기반
Predicting Camera Pose from Perspective Descriptions for Spatial Reasoning
Multi-image spatial reasoning remains challenging for current multimodal large language models (MLLMs). While single-view perception is inherently 2D, reasoning over multiple views requires building a coherent scene unde…
Scene UnderstandingSpatial ReasoningGeometry-Aware Neural Rendering
Understanding the 3-dimensional structure of the world is a core challenge in computer vision and robotics. Neural rendering approaches learn an implicit 3D model by predicting what a camera would see from an arbitrary v…
Neural RenderingGeometry-aware 4D Video Generation for Robot Manipulation
Understanding and predicting the dynamics of the physical world can enhance a robot's ability to plan and interact effectively in complex environments. While recent video generation models have shown strong potential in …
Robot ManipulationVideo GenerationFrom Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) po…
Viewpoint Matters: Dynamically Optimizing Viewpoints with Masked Autoencoder for Visual Manipulation
Robotic manipulation continues to be a challenge, and imitation learning (IL) enables robots to learn tasks from expert demonstrations. Current IL methods typically rely on fixed camera setups, where cameras are manually…