Photorealistic Image Synthesis for Object Instance Detection
We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images. The proposed approach has three key ingredients: (1) 3D object models are rendered in 3D models of complete scenes with realistic materials and lighting, (2) plausible geometric configuration of objects and cameras in a scene is generated using physics simulations, and (3) high photorealism of the synthesized images achieved by physically based rendering. When trained on images synthesized by the proposed approach, the Faster R-CNN object detector achieves a 24% absolute improvement of mAP@.75IoU on Rutgers APC and 11% on LineMod-Occluded datasets, compared to a baseline where the training images are synthesized by rendering object models on top of random photographs. This work is a step towards being able to effectively train object detectors without capturing or annotating any real images. A dataset of 600K synthetic images with ground truth annotations for various computer vision tasks will be released on the project website: thodan.github.io/objectsynth.
Code (0)
등록된 구현이 없습니다.
Tasks
6D Pose Estimation6D Pose Estimation using RGBImage GenerationObjectObject LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Panoptic-based Image Synthesis
Conditional image synthesis for generating photorealistic images serves various applications for content editing to content generation. Previous conditional image synthesis algorithms mostly rely on semantic maps, and of…
Image GenerationCountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidance
Diffusion models excel at photorealistic synthesis but struggle with precise object counts, especially in high-density settings. We introduce COUNTLOOP, a training-free framework that achieves precise instance control th…
Image GenerationObjectSplat: Improving Mesh Fidelity and Interactivity for 3D Scenes via Object-Level Mesh Splatting
Splatting-based algorithms reconstruct photorealistic, real-time-renderable, and mesh-exportable 3D scenes from regular images, but they represent a scene as a single monolithic field. Therefore, the reconstruction has n…
Detection and Segmentation of Custom Objects using High Distraction Photorealistic Synthetic Data
We show a straightforward and useful methodology for performing instance segmentation using synthetic data. We apply this methodology on a basic case and derived insights through quantitative analysis. We created a new p…
Instance SegmentationObject DetectionSegmentationSemantic SegmentationINeRF: Inverting Neural Radiance Fields for Pose Estimation
We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF). NeRFs have been shown to be remarkably effective for the task of view synthesis - synthesizing photoreal…
NeRFObjectPose Estimation