Driving with Regulation: Interpretable Decision-Making for Autonomous Vehicles with Retrieval-Augmented Reasoning via LLM
This work presents an interpretable decision-making framework for autonomous vehicles that integrates traffic regulations, norms, and safety guidelines comprehensively and enables seamless adaptation to different regions. While traditional rule-based methods struggle to incorporate the full scope of traffic rules, we develop a Traffic Regulation Retrieval (TRR) Agent based on Retrieval-Augmented Generation (RAG) to automatically retrieve relevant traffic rules and guidelines from extensive regulation documents and relevant records based on the ego vehicle's situation. Given the semantic complexity of the retrieved rules, we also design a reasoning module powered by a Large Language Model (LLM) to interpret these rules, differentiate between mandatory rules and safety guidelines, and assess actions on legal compliance and safety. Additionally, the reasoning is designed to be interpretable, enhancing both transparency and reliability. The framework demonstrates robust performance on both hypothesized and real-world cases across diverse scenarios, along with the ability to adapt to different regions with ease.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesDecision MakingLanguage ModelingLanguage ModellingLarge Language ModelRAGRetrievalRetrieval-augmented GenerationSimilar Papers 제목 키워드 기반
VLMs Guided Interpretable Decision Making for Autonomous Driving
Recent advancements in autonomous driving (AD) have explored the use of vision-language models (VLMs) within visual question answering (VQA) frameworks for direct driving decision-making. However, these approaches often …
Visual Question AnsweringScene UnderstandingAutonomous DrivingDecision MakingHybrid Reasoning Based on Large Language Models for Autonomous Car Driving
Large Language Models (LLMs) have garnered significant attention for their ability to understand text and images, generate human-like text, and perform complex reasoning tasks. However, their ability to generalize this a…
Autonomous DrivingAutonomous VehiclesCommon Sense ReasoningDecision MakingReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
End-to-end autonomous driving has emerged as a promising approach to unify perception, prediction, and planning within a single framework, reducing information loss and improving adaptability. However, existing methods o…
Autonomous DrivingA Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for end-to-end autonomous driving by jointly integrating perception, reasoning, and decision making within a unified multimodal framework. However, …
Visual Question AnsweringTrajectory PlanningAutonomous DrivingDecision MakingADRD: LLM-Driven Autonomous Driving Based on Rule-based Decision Systems
How to construct an interpretable autonomous driving decision-making system has become a focal point in academic research. In this study, we propose a novel approach that leverages large language models (LLMs) to generat…
Autonomous DrivingDecision Making