DR-KFS: A Differentiable Visual Similarity Metric for 3D Shape Reconstruction
We introduce a differential visual similarity metric to train deep neural networks for 3D reconstruction, aimed at improving reconstruction quality. The metric compares two 3D shapes by measuring distances between multi-view images differentiably rendered from the shapes. Importantly, the image-space distance is also differentiable and measures visual similarity, rather than pixel-wise distortion. Specifically, the similarity is defined by mean-squared errors over HardNet features computed from probabilistic keypoint maps of the compared images. Our differential visual shape similarity metric can be easily plugged into various 3D reconstruction networks, replacing their distortion-based losses, such as Chamfer or Earth Mover distances, so as to optimize the network weights to produce reconstructions with better structural fidelity and visual quality. We demonstrate this both objectively, using well-known shape metrics for retrieval and classification tasks that are independent from our new metric, and subjectively through a perceptual study.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Reconstruction3D Shape ReconstructionRetrievalSimilar Papers 제목 키워드 기반
Shape Reconstruction Using Differentiable Projections and Deep Priors
We investigate the problem of reconstructing shapes from noisy and incomplete projections in the presence of viewpoint uncertainities. The problem is cast as an optimization over the shape given measurements obtained by …
3D Shape ReconstructionLearning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
Conventional methods of 3D object generative modeling learn volumetric predictions using deep networks with 3D convolutional operations, which are direct analogies to classical 2D ones. However, these methods are computa…
3D Object ReconstructionObjectObject ReconstructionPoint Cloud GenerationFlexible Techniques for Differentiable Rendering with 3D Gaussians
Fast, reliable shape reconstruction is an essential ingredient in many computer vision applications. Neural Radiance Fields demonstrated that photorealistic novel view synthesis is within reach, but was gated by performa…
CPUGPUNovel View SynthesisOptical Flow EstimationDifferentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision
Learning-based 3D reconstruction methods have shown impressive results. However, most methods require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differenti…
3D ReconstructionMulti-View 3D ReconstructionShape As Points: A Differentiable Poisson Solver
In recent years, neural implicit representations gained popularity in 3D reconstruction due to their expressiveness and flexibility. However, the implicit nature of neural implicit representations results in slow inferen…
3D ReconstructionGPUSurface Reconstruction