Vision-Integrated LLMs for Autonomous Driving Assistance : Human Performance Comparison and Trust Evaluation
Traditional autonomous driving systems often struggle with reasoning in complex, unexpected scenarios due to limited comprehension of spatial relationships. In response, this study introduces a Large Language Model (LLM)-based Autonomous Driving (AD) assistance system that integrates a vision adapter and an LLM reasoning module to enhance visual understanding and decision-making. The vision adapter, combining YOLOv4 and Vision Transformer (ViT), extracts comprehensive visual features, while GPT-4 enables human-like spatial reasoning and response generation. Experimental evaluations with 45 experienced drivers revealed that the system closely mirrors human performance in describing situations and moderately aligns with human decisions in generating appropriate responses.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingDecision MakingLanguage ModelingLanguage ModellingLarge Language ModelResponse GenerationSpatial ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Vision-based Driver Assistance Systems: Survey, Taxonomy and Advances
Vision-based driver assistance systems is one of the rapidly growing research areas of ITS, due to various factors such as the increased level of safety requirements in automotive, computational power in embedded systems…
Autonomous DrivingSurveyTesting Large Language Models on Driving Theory Knowledge and Skills for Connected Autonomous Vehicles
Handling long tail corner cases is a major challenge faced by autonomous vehicles (AVs). While large language models (LLMs) hold great potentials to handle the corner cases with excellent generalization and explanation c…
Autonomous DrivingAutonomous VehiclesA Survey on Multimodal Large Language Models for Autonomous Driving
With the emergence of Large Language Models (LLMs) and Vision Foundation Models (VFMs), multimodal AI systems benefiting from large models have the potential to equally perceive the real world, make decisions, and contro…
Autonomous DrivingAdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
Effectively integrating Large Language Models (LLMs) into autonomous driving requires a balance between leveraging high-level reasoning and maintaining real-time efficiency. Existing approaches either activate LLMs too f…
Computational EfficiencyAutonomous DrivingRobustness of Segment Anything Model (SAM) for Autonomous Driving in Adverse Weather Conditions
Segment Anything Model (SAM) has gained considerable interest in recent times for its remarkable performance and has emerged as a foundational model in computer vision. It has been integrated in diverse downstream tasks,…
Autonomous Driving