Perspective Plane Program Induction from a Single Image
We study the inverse graphics problem of inferring a holistic representation for natural images. Given an input image, our goal is to induce a neuro-symbolic, program-like representation that jointly models camera poses, object locations, and global scene structures. Such high-level, holistic scene representations further facilitate low-level image manipulation tasks such as inpainting. We formulate this problem as jointly finding the camera pose and scene structure that best describe the input image. The benefits of such joint inference are two-fold: scene regularity serves as a new cue for perspective correction, and in turn, correct perspective correction leads to a simplified scene structure, similar to how the correct shape leads to the most regular texture in shape from texture. Our proposed framework, Perspective Plane Program Induction (P3I), combines search-based and gradient-based algorithms to efficiently solve the problem. P3I outperforms a set of baselines on a collection of Internet images, across tasks including camera pose estimation, global structure inference, and down-stream image manipulation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Camera Pose EstimationImage ManipulationPose EstimationProgram inductionShape from TextureSimilar Papers 제목 키워드 기반
Multi-Plane Program Induction with 3D Box Priors
We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene. Unlike prior work on image-based program…
Program inductionProgram SynthesisGuided Sketch-Based Program Induction by Search Gradients
Many tasks can be easily solved using machine learning techniques. However, some tasks cannot readily be solved using statistical models, requiring a symbolic approach instead. Program induction is one of the ways that s…
Program inductionUnposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model
Room layout estimation from multiple-perspective images is poorly investigated due to the complexities that emerge from multi-view geometry, which requires muti-step solutions such as camera intrinsic and extrinsic estim…
3D ReconstructionRoom Layout EstimationPlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
This paper proposes a deep neural network (DNN) for piece-wise planar depthmap reconstruction from a single RGB image. While DNNs have brought remarkable progress to single-image depth prediction, piece-wise planar depth…
Depth EstimationDepth PredictionSegmentationNeural Program Meta-Induction
Most recently proposed methods for Neural Program Induction work under the assumption of having a large set of input/output (I/O) examples for learning any underlying input-output mapping. This paper aims to address the …
Program inductionTransfer Learning