Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes
Recovering the 3D shape of transparent objects using a small number of unconstrained natural images is an ill-posed problem. Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this challenge. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera, under a known but arbitrary environment map. Our novel contributions include a normal representation that enables the network to model complex light transport through local computation, a rendering layer that models refractions and reflections, a cost volume specifically designed for normal refinement of transparent shapes and a feature mapping based on predicted normals for 3D point cloud reconstruction. We render a synthetic dataset to encourage the model to learn refractive light transport across different views. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images. Code and data are publicly released.
Code (1)
Tasks
3D geometry3D Point Cloud Reconstruction3D ReconstructionPoint cloud reconstructionTransparent objectsSimilar Papers 제목 키워드 기반
Fast 3D Reconstruction of Faces With Glasses
We present a method for the fast 3D face reconstruction of people wearing glasses. Our method explicitly and robustly models the case in which a face to be reconstructed is partially occluded by glasses. We propose a sim…
3D Face Reconstruction3D ReconstructionFace ReconstructionEyeglasses 3D shape reconstruction from a single face image
A complete 3D face reconstruction requires to explicitly model the eyeglasses on the face, which is less investigated in the literature. In this paper, we present an automatic system that recovers the 3D shape of eyeglas…
3D Face Reconstruction3D Reconstruction3D Shape ReconstructionFace Reconstruction+13D Reconstruction in the Presence of Glasses by Acoustic and Stereo Fusion
We present a practical and inexpensive method to reconstruct 3D scenes that include piece-wise planar transparent objects. Our work is motivated by the need for automatically generating 3D models of interior scenes, in w…
3D ReconstructionSensor FusionTransparent objectsSeeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container
In this paper, we define a new problem of recovering the 3D geometry of an object confined in a transparent enclosure. We also propose a novel method for solving this challenging problem. Transparent enclosures pose chal…
3D geometry3D ReconstructionGlassLoc: Plenoptic Grasp Pose Detection in Transparent Clutter
Transparent objects are prevalent across many environments of interest for dexterous robotic manipulation. Such transparent material leads to considerable uncertainty for robot perception and manipulation, and remains an…
Transparent objects