In the Driver's Seat: A Multi-Company Study on the Reality of Autonomous Driving System Testing
Autonomous driving systems (ADS) are rapidly advancing and increasingly deployed in real-world applications. This creates growing demands for effective testing to ensure system functionality and safety. However, ADS testing remains complex and lacks well-established standards for scenario selection, performance evaluation, and acceptance criteria. To better understand current ADS testing practices and challenges, we conducted an interview study with experts working on ADS development and testing in nine companies from six different countries. Through thematic analysis, we synthesized industrial testing practices, challenges, potential solutions, future trends, and proposed an evidence-centered closed-loop testing framework for ADS testing. Our findings show that current practices primarily focus on scenario-based and X-in-the-loop testing approaches, supported by diverse tools, metrics, benchmarks, and testing strategies. The participants highlighted major challenges related to scenario realism, scenario coverage, simulation fidelity, and acceptance criteria, while also discussing potential solutions such as the use of AI, world models, and end-to-end approaches. Furthermore, participants envisioned future ADS testing to become more automated, data-driven, and transparent across the industry. Overall, this study provides a comprehensive industry-grounded overview of ADS testing, proposes an evidence-centered closed-loop testing framework to provide actionable guidance for ADS testing, and outlines important directions for future research and practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingSimilar Papers 제목 키워드 기반
Robust Seatbelt Detection and Usage Recognition for Driver Monitoring Systems
Wearing a seatbelt appropriately while driving can reduce serious crash-related injuries or deaths by about half. However, current seatbelt reminder system has multiple shortcomings, such as can be easily fooled by a "Se…
NADS-Net: A Nimble Architecture for Driver and Seat Belt Detection via Convolutional Neural Networks
A new convolutional neural network (CNN) architecture for 2D driver/passenger pose estimation and seat belt detection is proposed in this paper. The new architecture is more nimble and thus more suitable for in-vehicle m…
Autonomous DrivingPose EstimationDriverseat: Crowdstrapping Learning Tasks for Autonomous Driving
While emerging deep-learning systems have outclassed knowledge-based approaches in many tasks, their application to detection tasks for autonomous technologies remains an open field for scientific exploration. Broadly, t…
Autonomous DrivingLane DetectionIntegrated In-vehicle Monitoring System Using 3D Human Pose Estimation and Seat Belt Segmentation
Recently, along with interest in autonomous vehicles, the importance of monitoring systems for both drivers and passengers inside vehicles has been increasing. This paper proposes a novel in-vehicle monitoring system the…
3D Human Pose Estimation3D Pose EstimationAutonomous VehiclesPose Estimation+1A case study of Consistent Vehicle Routing Problem with Time Windows
We develop a heuristic for the Consistent Vehicle Routing Problem with Time Windows (ConVRPTW), which is motivated by a real-world application at a food company's distribution center. Besides standard VRPTW restrictions,…