Learning to Rasterize Differentiably
Differentiable rasterization changes the standard formulation of primitive rasterization -- by enabling gradient flow from a pixel to its underlying triangles -- using distribution functions in different stages of rendering, creating a "soft" version of the original rasterizer. However, choosing the optimal softening function that ensures the best performance and convergence to a desired goal requires trial and error. Previous work has analyzed and compared several combinations of softening. In this work, we take it a step further and, instead of making a combinatorial choice of softening operations, parameterize the continuous space of common softening operations. We study meta-learning tunable softness functions over a set of inverse rendering tasks (2D and 3D shape, pose and occlusion) so it generalizes to new and unseen differentiable rendering tasks with optimal softness.
Code (1)
Tasks
Inverse RenderingMeta-LearningSimilar Papers 제목 키워드 기반
Rasterized Edge Gradients: Handling Discontinuities Differentiably
Computing the gradients of a rendering process is paramount for diverse applications in computer vision and graphics. However, accurate computation of these gradients is challenging due to discontinuities and rendering a…
Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks
We present a new versatile building block for deep point cloud processing architectures that is equally suited for diverse tasks. This building block combines the ideas of spatial transformers and multi-view convolutiona…
Point Cloud ClassificationPoint cloud reconstructionPoint Cloud SegmentationSemantic SegmentationCoordinate descent heuristics for the irregular strip packing problem of rasterized shapes
We consider the irregular strip packing problem of rasterized shapes, where a given set of pieces of irregular shapes represented in pixels should be placed into a rectangular container without overlap. The rasterized sh…
Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction
Rendering is the process of generating 2D images from 3D assets, simulated in a virtual environment, typically with a graphics pipeline. By inverting such renderer, one can think of a learning approach to predict a 3D sh…
Rasterized Steered Mixture of Experts for Efficient 2D Image Regression
The Steered Mixture of Experts regression framework has demonstrated strong performance in image reconstruction, compression, denoising, and super-resolution. However, its high computational cost limits practical applica…
Computational EfficiencyImage ReconstructionImage Denoising