Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images
We show that generative models can be used to capture visual geometry constraints statistically. We use this fact to infer the 3D shape of object categories from raw single-view images. Differently from prior work, we use no external supervision, nor do we use multiple views or videos of the objects. We achieve this by a simple reconstruction task, exploiting the symmetry of the objects' shape and albedo. Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder. Our experiments show that this method can recover the 3D shape of human faces, cat faces, and cars from single view images, without supervision. On benchmarks, we demonstrate superior accuracy compared to other methods that use supervision at the level of 2D image correspondences.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
We propose KeypointGAN, a new method for recognizing the pose of objects from a single image that for learning uses only unlabelled videos and a weak empirical prior on the object poses. Video frames differ primarily in …
Facial Landmark DetectionImage-to-Image TranslationKeypoint DetectionSelf-Supervised Learning+2GIST: Towards Photorealistic Style Transfer via Multiscale Geometric Representations
State-of-the-art Style Transfer methods often leverage pre-trained encoders optimized for discriminative tasks, which may not be ideal for image synthesis. This can result in significant artifacts and loss of photorealis…
Image GenerationStyle TransferMore Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs. Whilst the numbe…
AllCross-Modal RetrievalImage RetrievalRetrieval+2DiPE: Deeper into Photometric Errors for Unsupervised Learning of Depth and Ego-motion from Monocular Videos
Unsupervised learning of depth and ego-motion from unlabelled monocular videos has recently drawn great attention, which avoids the use of expensive ground truth in the supervised one. It achieves this by using the photo…
Autonomous DrivingMonocular Depth EstimationMotion EstimationGeometric Surface Image Prediction for Image Recognition Enhancement
This work presents a method to predict a geometric surface image from a photograph to assist in image recognition. To recognize objects, several images from different conditions are required for training a model or fine-…
Prediction