Accurate Human Body Reconstruction for Volumetric Video
In this work, we enhance a professional end-to-end volumetric video production pipeline to achieve high-fidelity human body reconstruction using only passive cameras. While current volumetric video approaches estimate depth maps using traditional stereo matching techniques, we introduce and optimize deep learning-based multi-view stereo networks for depth map estimation in the context of professional volumetric video reconstruction. Furthermore, we propose a novel depth map post-processing approach including filtering and fusion, by taking into account photometric confidence, cross-view geometric consistency, foreground masks as well as camera viewing frustums. We show that our method can generate high levels of geometric detail for reconstructed human bodies.
Code (0)
등록된 구현이 없습니다.
Tasks
Stereo MatchingVideo ReconstructionSimilar Papers 제목 키워드 기반
Deep Volumetric Video From Very Sparse Multi-View Performance Capture
We present a deep learning-based volumetric capture approach for performance capture using a passive and highly sparse multi-view capture system. We focus on a template-free, per-frame 3D surface reconstruction from as f…
Surface ReconstructionNeural Haircut: Prior-Guided Strand-Based Hair Reconstruction
Generating realistic human 3D reconstructions using image or video data is essential for various communication and entertainment applications. While existing methods achieved impressive results for body and facial region…
Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras
We aim to simultaneously estimate the 3D articulated pose and high fidelity volumetric occupancy of human performance, from multiple viewpoint video (MVV) with as few as two views. We use a multi-channel symmetric 3D con…
3D Human Pose EstimationDecoderPose EstimationGeneral Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model pe…
Markerless Motion CaptureBodyNet: Volumetric Inference of 3D Human Body Shapes
Human shape estimation is an important task for video editing, animation and fashion industry. Predicting 3D human body shape from natural images, however, is highly challenging due to factors such as variation in human …
3D Human Pose EstimationSegmentationVideo Editing