H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning
Parametric human models capture global pose but cannot represent the non-rigid surface dynamics of clothing and soft tissue. Generic scene flow estimates dense motion but breaks down on articulated bodies, where pixel-level supervision is also intractable to acquire. We introduce H-Flow, a dense human scene flow that captures both skeletal kinematics and surface deformation. A unified multi-head transformer estimates flow from monocular video, jointly predicting pose and depth as companion outputs. The challenge lies in the lack of supervision. In place of unattainable labels, we anchor the network in the physics of human motion, encoding geometric, structural, and biomechanical priors as cross-modal training objectives. We further introduce DynAct4D, a high-fidelity synthetic benchmark providing dense flow annotations across diverse subjects, garments, and motions. On standard benchmarks, H-Flow outperforms scene-flow and parametric baselines, and generalizes zero-shot to in-the-wild video. Code, models, and the DynAct4D benchmark will be released upon publication
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DoGFlow: Self-Supervised LiDAR Scene Flow via Cross-Modal Doppler Guidance
Accurate 3D scene flow estimation is critical for autonomous systems to navigate dynamic environments safely, but creating the necessary large-scale, manually annotated datasets remains a significant bottleneck for devel…
Scene Flow EstimationSelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction
Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely…
Autonomous DrivingLearning Optical Flow, Depth, and Scene Flow without Real-World Labels
Self-supervised monocular depth estimation enables robots to learn 3D perception from raw video streams. This scalable approach leverages projective geometry and ego-motion to learn via view synthesis, assuming the world…
Autonomous DrivingDepth EstimationMonocular Depth EstimationMulti-Task Learning+2RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior
In this work, we focus on scene flow learning on point clouds in a self-supervised manner. A real-world scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented a…
Motion EstimationSelf-Supervised LearningSelf-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion
Most of the current scene flow methods choose to model scene flow as a per point translation vector without differentiating between static and dynamic components of 3D motion. In this work we present an alternative metho…
Self-Supervised LearningTranslation