paper-with-me

Papers

Evolving Testing Scenario Generation Method and Intelligence Evaluation Framework for Automated Vehicles

2023-06-12 · Yining Ma, Wei Jiang, Lingtong Zhang, Junyi Chen, Hong Wang, Chen Lv, Xuesong Wang, Lu Xiong

Interaction between the background vehicles (BVs) and automated vehicles (AVs) in scenario-based testing plays a critical role in evaluating the intelligence of the AVs. Current testing scenarios typically employ predefined or scripted BVs, which inadequately reflect the complexity of human-like social behaviors in real-world driving scenarios, and also lack a systematic metric for evaluating the comprehensive intelligence of AVs. Therefore, this paper proposes an evolving scenario generation method that utilizes deep reinforcement learning (DRL) to create human-like BVs for testing and intelligence evaluation of AVs. Firstly, a class of driver models with human-like competitive, cooperative, and mutual driving motivations is designed. Then, utilizing an improved "level-k" training procedure, the three distinct driver models acquire game-based interactive driving policies. And these models are assigned to BVs for generating evolving scenarios in which all BVs can interact continuously and evolve diverse contents. Next, a framework including safety, driving efficiency, and interaction utility are presented to evaluate and quantify the intelligence performance of 3 systems under test (SUTs), indicating the effectiveness of the evolving scenario for intelligence testing. Finally, the complexity and fidelity of the proposed evolving testing scenario are validated. The results demonstrate that the proposed evolving scenario exhibits the highest level of complexity compared to other baseline scenarios and has more than 85% similarity to naturalistic driving data. This highlights the potential of the proposed method to facilitate the development and evaluation of high-level AVs in a realistic and challenging environment.

📄 PDF Abstract BibTeX arXiv:2306.07142

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learning

Similar Papers 제목 키워드 기반

An Evolving Scenario Generation Method based on Dual-modal Driver Model Trained by Multi-Agent Reinforcement Learning

2025-08-04 · Xinzheng Wu, Junyi Chen, Shaolingfeng Ye, Wei Jiang 외 arxiv

In the autonomous driving testing methods based on evolving scenarios, the construction method of the driver model, which determines the driving maneuvers of background vehicles (BVs) in the scenario, plays a critical ro…

Multi-agent Reinforcement LearningAutonomous Driving

Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World

2020-03-17 · Daniel J. Fremont, Edward Kim, Yash Vardhan Pant, Sanjit A. Seshia 외

We present a new approach to automated scenario-based testing of the safety of autonomous vehicles, especially those using advanced artificial intelligence-based components, spanning both simulation-based evaluation as w…

Autonomous Vehicles

Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness

2024-08-31 · Wenxuan Wang

Large language models (LLMs), such as ChatGPT, have rapidly penetrated into people's work and daily lives over the past few years, due to their extraordinary conversational skills and intelligence. ChatGPT has become the…

FairnessLanguage ModelingLanguage ModellingLogical Reasoning+2

Towards Closed-Loop Embodied Empathy Evolution: Probing LLM-Centric Lifelong Empathic Motion Generation in Unseen Scenarios

2025-12-22 · Jiawen Wang, Jingjing Wang Tianyang Chen, Min Zhang, Guodong Zhou arxiv

In the literature, existing human-centric emotional motion generation methods primarily focus on boosting performance within a single scale-fixed dataset, largely neglecting the flexible and scale-increasing motion scena…

TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework

2025-10-20 · Shuzheng Gao, Eric John Li, Man Ho Lam, Jingyu Xiao 외 arxiv

Large foundation models are fundamentally transforming the software engineering landscape, demonstrating exceptional capabilities across diverse tasks such as code generation, debugging, and testing. Despite this rapid p…

Code Generation