Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects
Our world is full of identical objects (\emphe.g., cans of coke, cars of same model). These duplicates, when seen together, provide additional and strong cues for us to effectively reason about 3D. Inspired by this observation, we introduce Structure from Duplicates (SfD), a novel inverse graphics framework that reconstructs geometry, material, and illumination from a single image containing multiple identical objects. SfD begins by identifying multiple instances of an object within an image, and then jointly estimates the 6DoF pose for all instances.An inverse graphics pipeline is subsequently employed to jointly reason about the shape, material of the object, and the environment light, while adhering to the shared geometry and material constraint across instances. Our primary contributions involve utilizing object duplicates as a robust prior for single-image inverse graphics and proposing an in-plane rotation-robust Structure from Motion (SfM) formulation for joint 6-DoF object pose estimation. By leveraging multi-view cues from a single image, SfD generates more realistic and detailed 3D reconstructions, significantly outperforming existing single image reconstruction models and multi-view reconstruction approaches with a similar or greater number of observations.
Code (1)
Tasks
Image ReconstructionObjectPose EstimationSimilar Papers 제목 키워드 기반
Re-Thinking Inverse Graphics With Large Language Models
Inverse graphics -- the task of inverting an image into physical variables that, when rendered, enable reproduction of the observed scene -- is a fundamental challenge in computer vision and graphics. Successfully disent…
Language ModellingLarge Language ModelSpatial ReasoningWorld Knowledge3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation
The ability to perceive and understand 3D scenes is crucial for many applications in computer vision and robotics. Inverse graphics is an appealing approach to 3D scene understanding that aims to infer the 3D scene struc…
6D Pose Estimation6D Pose Estimation using RGBPose EstimationPose Tracking+13DP3: 3D Scene Perception via Probabilistic Programming
We present 3DP3, a framework for inverse graphics that uses inference in a structured generative model of objects, scenes, and images. 3DP3 uses (i) voxel models to represent the 3D shape of objects, (ii) hierarchical sc…
ObjectPose EstimationProbabilistic ProgrammingScene UnderstandingPhysics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video
We propose a model that is able to perform unsupervised physical parameter estimation of systems from video, where the differential equations governing the scene dynamics are known, but labeled states or objects are not …
Inductive BiasModel Predictive Controlparameter estimationScene Understanding+1A Neural-Symbolic Architecture for Inverse Graphics Improved by Lifelong Meta-Learning
We follow the idea of formulating vision as inverse graphics and propose a new type of element for this task, a neural-symbolic capsule. It is capable of de-rendering a scene into semantic information feed-forward, as we…
Few-Shot LearningMeta-Learning