Learn-to-Race: A Multimodal Control Environment for Autonomous Racing
Existing research on autonomous driving primarily focuses on urban driving, which is insufficient for characterising the complex driving behaviour underlying high-speed racing. At the same time, existing racing simulation frameworks struggle in capturing realism, with respect to visual rendering, vehicular dynamics, and task objectives, inhibiting the transfer of learning agents to real-world contexts. We introduce a new environment, where agents Learn-to-Race (L2R) in simulated competition-style racing, using multimodal information--from virtual cameras to a comprehensive array of inertial measurement sensors. Our environment, which includes a simulator and an interfacing training framework, accurately models vehicle dynamics and racing conditions. In this paper, we release the Arrival simulator for autonomous racing. Next, we propose the L2R task with challenging metrics, inspired by learning-to-drive challenges, Formula-style racing, and multimodal trajectory prediction for autonomous driving. Additionally, we provide the L2R framework suite, facilitating simulated racing on high-precision models of real-world tracks. Finally, we provide an official L2R task dataset of expert demonstrations, as well as a series of baseline experiments and reference implementations. We make all code available: https://github.com/learn-to-race/l2r.
Code (1)
Tasks
Autonomous DrivingAutonomous RacingTrajectory PredictionSimilar Papers 제목 키워드 기반
The Edge of Disaster: A Battle Between Autonomous Racing and Safety
Autonomous racing represents a uniquely challenging control environment where agents must act while on the limits of a vehicle's capability in order to set competitive lap times. This places the agent on a knife's edge, …
Autonomous RacingModel Predictive ControlModel-Structured Neural Networks to Control the Steering Dynamics of Autonomous Race Cars
Autonomous racing has gained increasing attention in recent years, as a safe environment to accelerate the development of motion planning and control methods for autonomous driving. Deep learning models, predominantly ba…
Autonomous DrivingSteering ControlMotion PlanningVision based driving agent for race car simulation environments
In recent years, autonomous driving has become a popular field of study. As control at tire grip limit is essential during emergency situations, algorithms developed for racecars are useful for road cars too. This paper …
Autonomous DrivingDeep Reinforcement Learningreinforcement-learningReinforcement LearningSafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models
Traditional autonomous driving systems often struggle to connect high-level reasoning with low-level control, leading to suboptimal and sometimes unsafe behaviors. Recent advances in multimodal large language models (MLL…
AttributeAutonomous DrivingRAGRetrieval-augmented GenerationSense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing
Model-based reinforcement learning (MBRL) techniques have recently yielded promising results for real-world autonomous racing using high-dimensional observations. MBRL agents, such as Dreamer, solve long-horizon tasks by…
Autonomous Racingcontinuous-controlContinuous ControlModel-based Reinforcement Learning+2