IllumiNet: Transferring Illumination from Planar Surfaces to Virtual Objects in Augmented Reality
This paper presents an illumination estimation method for virtual objects in real environment by learning. While previous works tackled this problem by reconstructing high dynamic range (HDR) environment maps or the corresponding spherical harmonics, we do not seek to recover the lighting environment of the entire scene. Given a single RGB image, our method directly infers the relit virtual object by transferring the illumination features extracted from planar surfaces in the scene to the desired geometries. Compared to previous works, our approach is more robust as it works in both indoor and outdoor environments with spatially-varying illumination. Experiments and evaluation results show that our approach outperforms the state-of-the-art quantitatively and qualitatively, achieving realistic augmented experience.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Virtual Mirrors: Non-Line-of-Sight Imaging Beyond the Third Bounce
Non-line-of-sight (NLOS) imaging methods are capable of reconstructing complex scenes that are not visible to an observer using indirect illumination. However, they assume only third-bounce illumination, so they are curr…
DiLiGenT-Pi: Photometric Stereo for Planar Surfaces with Rich Details - Benchmark Dataset and Beyond
Photometric stereo aims to recover detailed surface shapes from images captured under varying illuminations. However, existing real-world datasets primarily focus on evaluating photometric stereo for general non-Lamb…
3D ReconstructionSurface Normal EstimationVIDIT: Virtual Image Dataset for Illumination Transfer
Deep image relighting is gaining more interest lately, as it allows photo enhancement through illumination-specific retouching without human effort. Aside from aesthetic enhancement and photo montage, image relighting is…
Domain AdaptationImage RelightingVehicle Models and Optimal Control on a Nonplanar Surface
We present a 10 DoF dynamic vehicle model for model-based control on nonplanar road surfaces. A parametric surface is used to describe the road surface, allowing the surface parameterization to describe the pose of the v…
Echo-Reconstruction: Audio-Augmented 3D Scene Reconstruction
Reflective and textureless surfaces such as windows, mirrors, and walls can be a challenge for object and scene reconstruction. These surfaces are often poorly reconstructed and filled with depth discontinuities and hole…
3D Reconstruction3D Scene ReconstructionClassificationDepth Estimation+1