RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation
Synthetic data is crucial for advancing autonomous driving (AD) systems, yet current state-of-the-art video generation models, despite their visual realism, suffer from subtle geometric distortions that limit their utility for downstream perception tasks. We identify and quantify this critical issue, demonstrating a significant performance gap in 3D object detection when using synthetic versus real data. To address this, we introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models. Its core components include an efficient Latent-Space Windowing Optimization technique for targeted feedback during diffusion, and a Hierarchical Geometric Reward (HGR) system providing multi-level rewards for point-line-plane alignment, and scene occupancy coherence. To quantify these distortions, we propose GeoScores. Applied to models like DiVE on nuScenes, RLGF substantially reduces geometric errors (e.g., VP error by 21\%, Depth error by 57\%) and dramatically improves 3D object detection mAP by 12.7\%, narrowing the gap to real-data performance. RLGF offers a plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning3D Object DetectionAutonomous DrivingVideo GenerationSimilar Papers 제목 키워드 기반
VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving
In autonomous driving, dynamic environment and corner cases pose significant challenges to the robustness of ego vehicle's state understanding and decision making. We introduce VDRive, a novel pipeline for end-to-end aut…
Reinforcement LearningAutonomous DrivingDecision MakingOptimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF
Reinforcement Learning from Human Feedback (RLHF) is popular in large language models (LLMs), whereas traditional Reinforcement Learning (RL) often falls short. Current autonomous driving methods typically utilize either…
Autonomous Drivingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Driving
In the field of autonomous driving, developing safe and trustworthy autonomous driving policies remains a significant challenge. Recently, Reinforcement Learning with Human Feedback (RLHF) has attracted substantial atten…
Autonomous DrivingPhilosophyreinforcement-learningReinforcement LearningLearning Personalized Discretionary Lane-Change Initiation for Fully Autonomous Driving Based on Reinforcement Learning
In this article, the authors present a novel method to learn the personalized tactic of discretionary lane-change initiation for fully autonomous vehicles through human-computer interactions. Instead of learning from hum…
Autonomous DrivingAutonomous VehiclesReinforcement Learning (RL)Finetuning Generative Trajectory Model with Reinforcement Learning from Human Feedback
Generating human-like and adaptive trajectories is essential for autonomous driving in dynamic environments. While generative models have shown promise in synthesizing feasible trajectories, they often fail to capture th…
Autonomous DrivingImitation LearningMotion PlanningNavSim