CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering
Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire. We explore the use of synthetic data for training CNN-based intrinsic image decomposition models, then applying these learned models to real-world images. To that end, we present \ICG, a new, large-scale dataset of physically-based rendered images of scenes with full ground truth decompositions. The rendering process we use is carefully designed to yield high-quality, realistic images, which we find to be crucial for this problem domain. We also propose a new end-to-end training method that learns better decompositions by leveraging \ICG, and optionally IIW and SAW, two recent datasets of sparse annotations on real-world images. Surprisingly, we find that a decomposition network trained solely on our synthetic data outperforms the state-of-the-art on both IIW and SAW, and performance improves even further when IIW and SAW data is added during training. Our work demonstrates the suprising effectiveness of carefully-rendered synthetic data for the intrinsic images task.
Code (0)
등록된 구현이 없습니다.
Tasks
Intrinsic Image DecompositionSimilar Papers 제목 키워드 기반
IDT: A Physically Grounded Transformer for Feed-Forward Multi-View Intrinsic Decomposition
Intrinsic image decomposition is fundamental for visual understanding, as RGB images entangle material properties, illumination, and view-dependent effects. Recent diffusion-based methods have achieved strong results for…
Single Image Intrinsic Decomposition without a Single Intrinsic Image
Intrinsic image decomposition---decomposing a natural image into a set of images corresponding to different physical causes---is one of the key and fundamental problems of computer vision. Previous intrinsic decompositio…
Intrinsic Image DecompositionDiscriminative feature encoding for intrinsic image decomposition
Intrinsic image decomposition is an important and long-standing computer vision problem. Given an input image, recovering the physical scene properties is ill-posed. Several physically motivated priors have been used to …
Intrinsic Image DecompositionPRISM: A Unified Framework for Photorealistic Reconstruction and Intrinsic Scene Modeling
We present PRISM, a unified framework that enables multiple image generation and editing tasks in a single foundational model. Starting from a pre-trained text-to-image diffusion model, PRISM proposes an effective fine-t…
Conditional Image GenerationImage GenerationIntrinsic Image DecompositionText to Image Generation+1Color naming guided intrinsic image decomposition
Intrinsic image decomposition is a severely under-constrained problem. User interactions can help to reduce the ambiguity of the decomposition considerably. The traditional way of user interaction is to draw scribbles th…
Color ConstancyIntrinsic Image Decomposition