DualAD: Dual-Layer Planning for Reasoning in Autonomous Driving
We present a novel autonomous driving framework, DualAD, designed to imitate human reasoning during driving. DualAD comprises two layers: a rule-based motion planner at the bottom layer that handles routine driving tasks requiring minimal reasoning, and an upper layer featuring a rule-based text encoder that converts driving scenarios from absolute states into text description. This text is then processed by a large language model (LLM) to make driving decisions. The upper layer intervenes in the bottom layer's decisions when potential danger is detected, mimicking human reasoning in critical situations. Closed-loop experiments demonstrate that DualAD, using a zero-shot pre-trained model, significantly outperforms rule-based motion planners that lack reasoning abilities. Our experiments also highlight the effectiveness of the text encoder, which considerably enhances the model's scenario understanding. Additionally, the integrated DualAD model improves with stronger LLMs, indicating the framework's potential for further enhancement. Code and benchmarks are available at github.com/TUM-AVS/DualAD.
Code (1)
Tasks
Autonomous DrivingLanguage ModelingLanguage ModellingLarge Language ModelSimilar Papers 제목 키워드 기반
DualAD: Disentangling the Dynamic and Static World for End-to-End Driving
State-of-the-art approaches for autonomous driving integrate multiple sub-tasks of the overall driving task into a single pipeline that can be trained in an end-to-end fashion by passing latent representations between th…
Autonomous DrivingCombining Adam and its Inverse Counterpart to Enhance Generalization of Deep Learning Optimizers
In the training of neural networks, adaptive moment estimation (Adam) typically converges fast but exhibits suboptimal generalization performance. A widely accepted explanation for its defect in generalization is that it…
Image ClassificationDo Agents Think Deeper? A Mechanistic Investigation of Layer-Wise Dynamics in Sequential Planning
Recent mechanistic studies suggest that large language models (LLMs) may utilize their depth inefficiently in standard single-turn tasks. Whether this still holds in autonomous agent settings, where models must perform m…
Code GenerationWebUncertainty: Dual-Level Uncertainty Driven Planning and Reasoning For Autonomous Web Agent
Recent advancements in large language models (LLMs) have empowered autonomous web agents to execute natural language instructions directly on real-world webpages. However, existing agents often struggle with complex task…
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models
A primary hurdle of autonomous driving in urban environments is understanding complex and long-tail scenarios, such as challenging road conditions and delicate human behaviors. We introduce DriveVLM, an autonomous drivin…
Autonomous DrivingScene UnderstandingSpatial Reasoning