Virtual to Real Reinforcement Learning for Autonomous Driving
Reinforcement learning is considered as a promising direction for driving policy learning. However, training autonomous driving vehicle with reinforcement learning in real environment involves non-affordable trial-and-error. It is more desirable to first train in a virtual environment and then transfer to the real environment. In this paper, we propose a novel realistic translation network to make model trained in virtual environment be workable in real world. The proposed network can convert non-realistic virtual image input into a realistic one with similar scene structure. Given realistic frames as input, driving policy trained by reinforcement learning can nicely adapt to real world driving. Experiments show that our proposed virtual to real (VR) reinforcement learning (RL) works pretty well. To our knowledge, this is the first successful case of driving policy trained by reinforcement learning that can adapt to real world driving data.
Code (6)
Tasks
Autonomous DrivingDomain AdaptationImage-to-Image Translationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Synthetic-to-Real TranslationTransfer LearningTranslationSimilar Papers 제목 키워드 기반
Autonomous Driving in Reality with Reinforcement Learning and Image Translation
Supervised learning is widely used in training autonomous driving vehicle. However, it is trained with large amount of supervised labeled data. Reinforcement learning can be trained without abundant labeled data, but we …
Autonomous DrivingCar Racingreinforcement-learningReinforcement Learning+3Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving
Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based met…
Autonomous DrivingAutonomous VehiclesDeep Reinforcement LearningMixed Reality+3Self-driving scale car trained by Deep reinforcement learning
The self-driving based on deep reinforcement learning, as the most important application of artificial intelligence, has become a popular topic. Most of the current self-driving methods focus on how to directly learn end…
Autonomous DrivingDeep Reinforcement LearningQ-Learningreinforcement-learning+3Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses
In the vehicular mixed reality (MR) Metaverse, the distance between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous drivin…
Autonomous DrivingAutonomous VehiclesMixed RealityUncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving
To further improve the learning efficiency and performance of reinforcement learning (RL), in this paper we propose a novel uncertainty-aware model-based RL (UA-MBRL) framework, and then implement and validate it in auto…
Autonomous DrivingModel-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)