A Scenario-Based Platform for Testing Autonomous Vehicle Behavior Prediction Models in Simulation
Behavior prediction remains one of the most challenging tasks in the autonomous vehicle (AV) software stack. Forecasting the future trajectories of nearby agents plays a critical role in ensuring road safety, as it equips AVs with the necessary information to plan safe routes of travel. However, these prediction models are data-driven and trained on data collected in real life that may not represent the full range of scenarios an AV can encounter. Hence, it is important that these prediction models are extensively tested in various test scenarios involving interactive behaviors prior to deployment. To support this need, we present a simulation-based testing platform which supports (1) intuitive scenario modeling with a probabilistic programming language called Scenic, (2) specifying a multi-objective evaluation metric with a partial priority ordering, (3) falsification of the provided metric, and (4) parallelization of simulations for scalable testing. As a part of the platform, we provide a library of 25 Scenic programs that model challenging test scenarios involving interactive traffic participant behaviors. We demonstrate the effectiveness and the scalability of our platform by testing a trained behavior prediction model and searching for failure scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionProbabilistic ProgrammingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
VP-AutoTest: A Virtual-Physical Fusion Autonomous Driving Testing Platform
The rapid development of autonomous vehicles has led to a surge in testing demand. Traditional testing methods, such as virtual simulation, closed-course, and public road testing, face several challenges, including unrea…
Autonomous VehiclesAutonomous DrivingInteractive Adversarial Testing of Autonomous Vehicles with Adjustable Confrontation Intensity
Scientific testing techniques are essential for ensuring the safe operation of autonomous vehicles (AVs), with high-risk, highly interactive scenarios being a primary focus. To address the limitations of existing testing…
Adversarial RobustnessAutonomous VehiclesA Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios
In recent years, control under urban intersection scenarios becomes an emerging research topic. In such scenarios, the autonomous vehicle confronts complicated situations since it must deal with the interaction with soci…
Autonomous Drivingreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Hierarchical Pedestrian Behavior Model to Generate Realistic Human Behavior in Traffic Simulation
Modelling pedestrian behavior is crucial in the development and testing of autonomous vehicles. In this work, we present a hierarchical pedestrian behavior model that generates high-level decisions through the use of beh…
Autonomous VehiclesDecision MakingA Learning-based Stochastic Driving Model for Autonomous Vehicle Testing
In the simulation-based testing and evaluation of autonomous vehicles (AVs), how background vehicles (BVs) drive directly influences the AV's driving behavior and further impacts the testing result. Existing simulation p…
Autonomous Vehiclesquantile regression