Textureless Deformable Surface Reconstruction with Invisible Markers
Reconstructing and tracking deformable surface with little or no texture has posed long-standing challenges. Fundamentally, the challenges stem from textureless surfaces lacking features for establishing cross-image correspondences. In this work, we present a novel type of markers to proactively enrich the object's surface features, and thereby ease the 3D surface reconstruction and correspondence tracking. Our markers are made of fluorescent dyes, visible only under the ultraviolet (UV) light and invisible under regular lighting condition. Leveraging the markers, we design a multi-camera system that captures surface deformation under the UV light and the visible light in a time multiplexing fashion. Under the UV light, markers on the object emerge to enrich its surface texture, allowing high-quality 3D shape reconstruction and tracking. Under the visible light, markers become invisible, allowing us to capture the object's original untouched appearance. We perform experiments on various challenging scenes, including hand gestures, facial expressions, waving cloth, and hand-object interaction. In all these cases, we demonstrate that our system is able to produce robust, high-quality 3D reconstruction and tracking.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Reconstruction3D Shape ReconstructionSurface ReconstructionSimilar Papers 제목 키워드 기반
PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Image
The misaligned human texture across different human parts is one of the main limitations of existing 3D human reconstruction methods. Each human part, such as a jacket or pants, should maintain a distinct texture without…
3D Human Reconstruction3D Part SegmentationImage GenerationPMNI: Pose-free Multi-view Normal Integration for Reflective and Textureless Surface Reconstruction
Reflective and textureless surfaces remain a challenge in multi-view 3D reconstruction.Both camera pose calibration and shape reconstruction often fail due to insufficient or unreliable cross-view visual features. To add…
Surface ReconstructionPISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects
Neural implicit surface reconstruction has achieved remarkable progress recently. Despite resorting to complex radiance modeling, state-of-the-art methods still struggle with textureless and specular surfaces. Different …
Surface ReconstructionDeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers
Fiducial markers have been broadly used to identify objects or embed messages that can be detected by a camera. Primarily, existing detection methods assume that markers are printed on ideally planar surfaces. Markers of…
Multi-View Neural Surface Reconstruction with Structured Light
Three-dimensional (3D) object reconstruction based on differentiable rendering (DR) is an active research topic in computer vision. DR-based methods minimize the difference between the rendered and target images by optim…
3D Object ReconstructionObject ReconstructionSurface Reconstruction