Learning to Infer 3D Shape Programs with Differentiable Renderer
Given everyday artifacts, such as tables and chairs, humans recognize high-level regularities within them, such as the symmetries of a table, the repetition of its legs, while possessing low-level priors of their geometries, e.g., surfaces are smooth and edges are sharp. This kind of knowledge constitutes an important part of human perceptual understanding and reasoning. Representations of and how to reason in such knowledge, and the acquisition thereof, are still open questions in artificial intelligence (AI) and cognitive science. Building on the previous proposal of the \emph{3D shape programs} representation alone with the accompanying neural generator and executor from \citet{tian2019learning}, we propose an analytical yet differentiable executor that is more faithful and controllable in interpreting shape programs (particularly in extrapolation) and more sample efficient (requires no training). These facilitate the generator's learning when ground truth programs are not available, and should be especially useful when new shape-program components are enrolled either by human designers or -- in the context of library learning -- algorithms themselves. Preliminary experiments on using it for adaptation illustrate the aforesaid advantages of the proposed module, encouraging similar methods being explored in building machines that learn to reason with the kind of knowledge described above, and even learn this knowledge itself.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GenDR: A Generalized Differentiable Renderer
In this work, we present and study a generalized family of differentiable renderers. We discuss from scratch which components are necessary for differentiable rendering and formalize the requirements for each component. …
3D ReconstructionInverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
Recent work has shown the ability to learn generative models for 3D shapes from only unstructured 2D images. However, training such models requires differentiating through the rasterization step of the rendering process,…
Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
Rendering bridges the gap between 2D vision and 3D scenes by simulating the physical process of image formation. By inverting such renderer, one can think of a learning approach to infer 3D information from 2D images. Ho…
3D Object ReconstructionSingle-View 3D ReconstructionApproximate Differentiable Rendering with Algebraic Surfaces
Differentiable renderers provide a direct mathematical link between an object's 3D representation and images of that object. In this work, we develop an approximate differentiable renderer for a compact, interpretable re…
Pose EstimationLearning to Infer and Execute 3D Shape Programs
Human perception of 3D shapes goes beyond reconstructing them as a set of points or a composition of geometric primitives: we also effortlessly understand higher-level shape structure such as the repetition and reflectiv…