MyGo: Consistent and Controllable Multi-View Driving Video Generation with Camera Control
High-quality driving video generation is crucial for providing training data for autonomous driving models. However, current generative models rarely focus on enhancing camera motion control under multi-view tasks, which is essential for driving video generation. Therefore, we propose MyGo, an end-to-end framework for video generation, introducing motion of onboard cameras as conditions to make progress in camera controllability and multi-view consistency. MyGo employs additional plug-in modules to inject camera parameters into the pre-trained video diffusion model, which retains the extensive knowledge of the pre-trained model as much as possible. Furthermore, we use epipolar constraints and neighbor view information during the generation process of each view to enhance spatial-temporal consistency. Experimental results show that MyGo has achieved state-of-the-art results in both general camera-controlled video generation and multi-view driving video generation tasks, which lays the foundation for more accurate environment simulation in autonomous driving. Project page: https://metadrivescape.github.io/papers_project/MyGo/page.html
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingVideo GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DrivingGaussian++: Towards Realistic Reconstruction and Editable Simulation for Surrounding Dynamic Driving Scenes
We present DrivingGaussian++, an efficient and effective framework for realistic reconstructing and controllable editing of surrounding dynamic autonomous driving scenes. DrivingGaussian++ models the static background us…
Autonomous DrivingBEV-VAE: Multi-view Image Generation with Spatial Consistency for Autonomous Driving
Multi-view image generation in autonomous driving demands consistent 3D scene understanding across camera views. Most existing methods treat this problem as a 2D image set generation task, lacking explicit 3D modeling. H…
Scene UnderstandingAutonomous DrivingScene GenerationImage GenerationV2VCrafter: Consistent Street-View Image Generation Across Vehicles
Connected and autonomous driving (CAD) systems leverage vehicle-to-vehicle (V2V) communication for multi-agent collaborative perception, yet remain constrained by scarce annotated real-world V2V datasets and limited gene…
3D Object DetectionData AugmentationImage GenerationDriving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving
In autonomous driving, predicting future events in advance and evaluating the foreseeable risks empowers autonomous vehicles to better plan their actions, enhancing safety and efficiency on the road. To this end, we prop…
Autonomous DrivingAutonomous VehiclesCoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Driving
Recent progress in driving video generation has shown significant potential for enhancing self-driving systems by providing scalable and controllable training data. Although pretrained state-of-the-art generation models,…
3D GenerationAutonomous DrivingVideo Generation