DreaMo: Articulated 3D Reconstruction From A Single Casual Video
Articulated 3D reconstruction has valuable applications in various domains, yet it remains costly and demands intensive work from domain experts. Recent advancements in template-free learning methods show promising results with monocular videos. Nevertheless, these approaches necessitate a comprehensive coverage of all viewpoints of the subject in the input video, thus limiting their applicability to casually captured videos from online sources. In this work, we study articulated 3D shape reconstruction from a single and casually captured internet video, where the subject's view coverage is incomplete. We propose DreaMo that jointly performs shape reconstruction while solving the challenging low-coverage regions with view-conditioned diffusion prior and several tailored regularizations. In addition, we introduce a skeleton generation strategy to create human-interpretable skeletons from the learned neural bones and skinning weights. We conduct our study on a self-collected internet video collection characterized by incomplete view coverage. DreaMo shows promising quality in novel-view rendering, detailed articulated shape reconstruction, and skeleton generation. Extensive qualitative and quantitative studies validate the efficacy of each proposed component, and show existing methods are unable to solve correct geometry due to the incomplete view coverage.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Reconstruction3D Shape ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generalizable Articulated Object Reconstruction from Casually Captured RGBD Videos
Articulated objects are prevalent in daily life. Understanding their kinematic structure and reconstructing them have numerous applications in embodied AI and robotics. However, current methods require carefully captured…
ObjectObject ReconstructionBANMo: Building Animatable 3D Neural Models from Many Casual Videos
Prior work for articulated 3D shape reconstruction often relies on specialized sensors (e.g., synchronized multi-camera systems), or pre-built 3D deformable models (e.g., SMAL or SMPL). Such methods are not able to scale…
3D Shape Reconstruction from VideosDynamic ReconstructionPF-LHM: 3D Animatable Avatar Reconstruction from Pose-free Articulated Human Images
Reconstructing an animatable 3D human from casually captured images of an articulated subject without camera or human pose information is a practical yet challenging task due to view misalignment, occlusions, and the abs…
3D Human ReconstructionImage ReconstructionPose EstimationArticulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Video
Retrieving the 3D kinematics of articulated objects from monocular video is a fundamental challenge in computer vision. Existing methods rely on complex video setups or cues such as long-term point tracking or wide-basel…
Point TrackingDreaMoving: A Human Video Generation Framework based on Diffusion Models
In this paper, we present DreaMoving, a diffusion-based controllable video generation framework to produce high-quality customized human videos. Specifically, given target identity and posture sequences, DreaMoving can g…
Video Generation