Driving Everywhere with Large Language Model Policy Adaptation
Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs). In this paper, we present LLaDA, a simple yet powerful tool that enables human drivers and autonomous vehicles alike to drive everywhere by adapting their tasks and motion plans to traffic rules in new locations. LLaDA achieves this by leveraging the impressive zero-shot generalizability of large language models (LLMs) in interpreting the traffic rules in the local driver handbook. Through an extensive user study, we show that LLaDA's instructions are useful in disambiguating in-the-wild unexpected situations. We also demonstrate LLaDA's ability to adapt AV motion planning policies in real-world datasets; LLaDA outperforms baseline planning approaches on all our metrics. Please check our website for more details: https://boyiliee.github.io/llada.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingAutonomous VehiclesLanguage ModelingLanguage ModellingLarge Language ModelmodelMotion PlanningSimilar Papers 제목 키워드 기반
LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Models
Recent advancements in reinforcement learning (RL) demonstrate the significant potential in autonomous driving. Despite this promise, challenges such as the manual design of reward functions and low sample efficiency in …
Autonomous DrivingLarge Language ModelReinforcement Learning (RL)Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving
The integration of Large Language Models (LLMs) into autonomous driving systems demonstrates strong common sense and reasoning abilities, effectively addressing the pitfalls of purely data-driven methods. Current LLM-bas…
Autonomous DrivingCommon Sense ReasoningReinforcement Learning (RL)Robust Object Detection for Autonomous Driving via Curriculum-Guided Group Relative Policy Optimization
Multimodal Large Language Models (MLLMs) excel in vision-language reasoning but often struggle with structured perception tasks requiring precise localization and robustness. We propose a reinforcement learning framework…
Robust Object DetectionReinforcement LearningAutonomous DrivingEfficient Policy Adaptation with Contrastive Prompt Ensemble for Embodied Agents
For embodied reinforcement learning (RL) agents interacting with the environment, it is desirable to have rapid policy adaptation to unseen visual observations, but achieving zero-shot adaptation capability is considered…
Autonomous DrivingLanguage ModelingLanguage ModellingReinforcement Learning (RL)WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving
End-to-end autonomous driving systems based on vision-language-action (VLA) models integrate multimodal sensor inputs and language instructions to generate planning and control signals. While autoregressive large languag…
Visual Question AnsweringReinforcement LearningAutonomous Driving