Simulation-based reinforcement learning for real-world autonomous driving
We use reinforcement learning in simulation to obtain a driving system controlling a full-size real-world vehicle. The driving policy takes RGB images from a single camera and their semantic segmentation as input. We use mostly synthetic data, with labelled real-world data appearing only in the training of the segmentation network. Using reinforcement learning in simulation and synthetic data is motivated by lowering costs and engineering effort. In real-world experiments we confirm that we achieved successful sim-to-real policy transfer. Based on the extensive evaluation, we analyze how design decisions about perception, control, and training impact the real-world performance.
Code (1)
Tasks
Autonomous Drivingreinforcement-learningReinforcement LearningReinforcement Learning (RL)SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning
We present TreeIRL, a novel planner for autonomous driving that combines Monte Carlo tree search (MCTS) and inverse reinforcement learning (IRL) to achieve state-of-the-art performance in simulation and in real-world dri…
Reinforcement LearningAutonomous DrivingSim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
Autonomous driving is complex, requiring sophisticated 3D scene understanding, localization, mapping, and control. Rather than explicitly modelling and fusing each of these components, we instead consider an end-to-end a…
Autonomous DrivingGPUReinforcement Learning (RL)Robot Manipulation+1Tackling Real-World Autonomous Driving using Deep Reinforcement Learning
In the typical autonomous driving stack, planning and control systems represent two of the most crucial components in which data retrieved by sensors and processed by perception algorithms are used to implement a safe an…
Autonomous DrivingDeep Reinforcement Learningreinforcement-learningReinforcement Learning+1ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstruction
Reinforcement learning for training end-to-end autonomous driving models in closed-loop simulations is gaining growing attention. However, most simulation environments differ significantly from real-world conditions, cre…
Reinforcement LearningAutonomous DrivingCIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving
Modern approaches to autonomous driving rely heavily on learned components trained with large amounts of human driving data via imitation learning. However, these methods require large amounts of expensive data collectio…
Autonomous DrivingAutonomous VehiclesImitation Learningreinforcement-learning+3