SIDNet: Learning Shading-aware Illumination Descriptor for Image Harmonization
Image harmonization aims at adjusting the appearance of the foreground to make it more compatible with the background. Without exploring background illumination and its effects on the foreground elements, existing works are incapable of generating a realistic foreground shading. In this paper, we decompose the image harmonization task into two sub-problems: 1) illumination estimation of the background image and 2) re-rendering of foreground objects under background illumination. Before solving these two sub-problems, we first learn a shading-aware illumination descriptor via a well-designed neural rendering framework, of which the key is a shading bases module that generates multiple shading bases from the foreground image. Then we design a background illumination estimation module to extract the illumination descriptor from the background. Finally, the Shading-aware Illumination Descriptor is used in conjunction with the neural rendering framework (SIDNet) to produce the harmonized foreground image containing a novel harmonized shading. Moreover, we construct a photo-realistic synthetic image harmonization dataset that contains numerous shading variations with image-based lighting. Extensive experiments on both synthetic and real data demonstrate the superiority of the proposed method, especially in dealing with foreground shadings.
Code (0)
등록된 구현이 없습니다.
Tasks
Image HarmonizationNeural RenderingSimilar Papers 제목 키워드 기반
Invariant Descriptors for Intrinsic Reflectance Optimization
Intrinsic image decomposition aims to factorize an image into albedo (reflectance) and shading (illumination) sub-components. Being ill-posed and under-constrained, it is a very challenging computer vision problem. There…
Intrinsic Image DecompositionA Variational Approach to Shape-from-shading Under Natural Illumination
A numerical solution to shape-from-shading under natural illumination is presented. It builds upon an augmented Lagrangian approach for solving a generic PDE-based shape-from-shading model which handles directional or sp…
DenoisingColorful Diffuse Intrinsic Image Decomposition in the Wild
Intrinsic image decomposition aims to separate the surface reflectance and the effects from the illumination given a single photograph. Due to the complexity of the problem, most prior works assume a single-color illumin…
Color ConstancyIntrinsic Image DecompositionInverse RenderingSpecular Reflection MitigationConsistency-aware Shading Orders Selective Fusion for Intrinsic Image Decomposition
We address the problem of decomposing a single image into reflectance and shading. The difficulty comes from the fact that the components of image---the surface albedo, the direct illumination, and the ambient illuminati…
Intrinsic Image DecompositionPIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition
Intrinsic image decomposition is the process of recovering the image formation components (reflectance and shading) from an image. Previous methods employ either explicit priors to constrain the problem or implicit const…
Intrinsic Image Decomposition