NeRD++: Improved 3D-mirror symmetry learning from a single image
Many objects are naturally symmetric, and this symmetry can be exploited to infer unseen 3D properties from a single 2D image. Recently, NeRD is proposed for accurate 3D mirror plane estimation from a single image. Despite the unprecedented accuracy, it relies on large annotated datasets for training and suffers from slow inference. Here we aim to improve its data and compute efficiency. We do away with the computationally expensive 4D feature volumes and instead explicitly compute the feature correlation of the pixel correspondences across depth, thus creating a compact 3D volume. We also design multi-stage spherical convolutions to identify the optimal mirror plane on the hemisphere, whose inductive bias offers gains in data-efficiency. Experiments on both synthetic and real-world datasets show the benefit of our proposed changes for improved data efficiency and inference speed.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature CorrelationInductive BiasSymmetry DetectionSimilar Papers 제목 키워드 기반
NeRD: Neural 3D Reflection Symmetry Detector
Recent advances have shown that symmetry, a structural prior that most objects exhibit, can support a variety of single-view 3D understanding tasks. However, detecting 3D symmetry from an image remains a challenging task…
Pose EstimationregressionReconstructing 3D Human Pose by Watching Humans in the Mirror
In this paper, we introduce the new task of reconstructing 3D human pose from a single image in which we can see the person and the person's image through a mirror. Compared to general scenarios of 3D pose estimation fro…
3D Pose EstimationPose EstimationPixNerd: Pixel Neural Field Diffusion
The current success of diffusion transformers heavily depends on the compressed latent space shaped by the pre-trained variational autoencoder(VAE). However, this two-stage training paradigm inevitably introduces accumul…
Symmetry-Aware Transformer-based Mirror Detection
Mirror detection aims to identify the mirror regions in the given input image. Existing works mainly focus on integrating the semantic features and structural features to mine specific relations between mirror and non-mi…
DecoderMirror DetectionNeRD: a Neural Response Divergence Approach to Visual Salience Detection
In this paper, a novel approach to visual salience detection via Neural Response Divergence (NeRD) is proposed, where synaptic portions of deep neural networks, previously trained for complex object recognition, are leve…
Object Recognition