Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
We present the first approach for 3D point-cloud to image translation based on conditional Generative Adversarial Networks (cGAN). The model handles multi-modal information sources from different domains, i.e. raw point-sets and images. The generator is capable of processing three conditions, whereas the point-cloud is encoded as raw point-set and camera projection. An image background patch is used as constraint to bias environmental texturing. A global approximation function within the generator is directly applied on the point-cloud (Point-Net). Hence, the representative learning model incorporates global 3D characteristics directly at the latent feature space. Conditions are used to bias the background and the viewpoint of the generated image. This opens up new ways in augmenting or texturing 3D data to aim the generation of fully individual images. We successfully evaluated our method on the Kitti and SunRGBD dataset with an outstanding object detection inception score.
Code (0)
등록된 구현이 없습니다.
Tasks
object-detectionObject DetectionTranslationSimilar Papers 제목 키워드 기반
A Conditional Generative Adversarial Network for Rendering Point Clouds
In computer graphics, point clouds from laser scanning devices are difficult to render into photo-realistic images due to lack of information they carry about color, normal, lighting, and connection between points. Rende…
Generative Adversarial NetworkSurface ReconstructionHow Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines
Visual localization is the problem of estimating the camera pose of a given image with respect to a known scene. Visual localization algorithms are a fundamental building block in advanced computer vision applications, i…
Camera Pose EstimationPose EstimationPrivacy PreservingTranslation+1Z2P: Instant Visualization of Point Clouds
We present a technique for visualizing point clouds using a neural network. Our technique allows for an instant preview of any point cloud, and bypasses the notoriously difficult surface reconstruction problem or the nee…
Image-to-Image TranslationSurface ReconstructionTranslationDepth-Guided Privacy-Preserving Visual Localization Using 3D Sphere Clouds
The emergence of deep neural networks capable of revealing high-fidelity scene details from sparse 3D point clouds has raised significant privacy concerns in visual localization involving private maps. Lifting map points…
Camera Pose EstimationVisual LocalizationPoint CloudsPoint Convolutional Neural Networks by Extension Operators
This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying convolutional neural networks to point clouds. The framework consists of two operators: extension and restriction, mapping po…
3D Part Segmentation3D Point Cloud ClassificationClassify 3D Point CloudsTranslation