RayTracer.jl: A Differentiable Renderer that supports Parameter Optimization for Scene Reconstruction
In this paper, we present RayTracer.jl, a renderer in Julia that is fully differentiable using source-to-source Automatic Differentiation (AD). This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.
Code (1)
Similar Papers 제목 키워드 기반
Plateau-reduced Differentiable Path Tracing
Current differentiable renderers provide light transport gradients with respect to arbitrary scene parameters. However, the mere existence of these gradients does not guarantee useful update steps in an optimization. Ins…
Inverse RenderingAnalytical Derivatives for Differentiable Renderer: 3D Pose Estimation by Silhouette Consistency
Differentiable render is widely used in optimization-based 3D reconstruction which requires gradients from differentiable operations for gradient-based optimization. The existing differentiable renderers obtain the gradi…
3D Pose Estimation3D ReconstructionPose EstimationInverting Generative Adversarial Renderer for Face Reconstruction
Given a monocular face image as input, 3D face geometry reconstruction aims to recover a corresponding 3D face mesh. Recently, both optimization-based and learning-based face reconstruction methods have taken advantage o…
Face ReconstructionGaussian Mesh Renderer for Lightweight Differentiable Rendering
3D Gaussian Splatting (3DGS) has enabled high-fidelity virtualization with fast rendering and optimization for novel view synthesis. On the other hand, triangle mesh models still remain a popular choice for surface recon…
Novel View SynthesisDIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer
We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiable renderers. Many previous learning-based approaches for inverse graphics adopt rasterization-b…
Disentanglement