SyntheOcc: Synthesize Geometric-Controlled Street View Images through 3D Semantic MPIs
The advancement of autonomous driving is increasingly reliant on high-quality annotated datasets, especially in the task of 3D occupancy prediction, where the occupancy labels require dense 3D annotation with significant human effort. In this paper, we propose SyntheOcc, which denotes a diffusion model that Synthesize photorealistic and geometric-controlled images by conditioning Occupancy labels in driving scenarios. This yields an unlimited amount of diverse, annotated, and controllable datasets for applications like training perception models and simulation. SyntheOcc addresses the critical challenge of how to efficiently encode 3D geometric information as conditional input to a 2D diffusion model. Our approach innovatively incorporates 3D semantic multi-plane images (MPIs) to provide comprehensive and spatially aligned 3D scene descriptions for conditioning. As a result, SyntheOcc can generate photorealistic multi-view images and videos that faithfully align with the given geometric labels (semantics in 3D voxel space). Extensive qualitative and quantitative evaluations of SyntheOcc on the nuScenes dataset prove its effectiveness in generating controllable occupancy datasets that serve as an effective data augmentation to perception models.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingData AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Geometry-Guided Street-View Panorama Synthesis from Satellite Imagery
This paper presents a new approach for synthesizing a novel street-view panorama given an overhead satellite image. Taking a small satellite image patch as input, our method generates a Google's omnidirectional street-vi…
Image GenerationGeometry-Aware Satellite-to-Ground Image Synthesis for Urban Areas
We present a novel method for generating panoramic street-view images which are geometrically consistent with a given satellite image. Different from existing approaches that completely rely on a deep learning architectu…
Image GenerationSat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Image
We present a novel method for synthesizing both temporally and geometrically consistent street-view panoramic video from a single satellite image and camera trajectory. Existing cross-view synthesis approaches focus on i…
Image GenerationUrban Radiance Fields
The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e.g., Street View). Given a seque…
3D ReconstructionNeRFNovel View SynthesisFloorLevel-Net: Recognizing Floor-Level Lines with Height-Attention-Guided Multi-task Learning
The ability to recognize the position and order of the floor-level lines that divide adjacent building floors can benefit many applications, for example, urban augmented reality (AR). This work tackles the problem of loc…
Data AugmentationMulti-Task Learning