paper-with-me

홈 › Papers

TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning

2025-02-03 · Chengkai Xu, Jiaqi Liu, Shiyu Fang, Yiming Cui, Dong Chen, Peng Hang, Jian Sun

Although Deep Reinforcement Learning (DRL) and Large Language Models (LLMs) each show promise in addressing decision-making challenges in autonomous driving, DRL often suffers from high sample complexity, while LLMs have difficulty ensuring real-time decision making. To address these limitations, we propose TeLL-Drive, a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy. By incorporating risk metrics, historical scenario retrieval, and domain heuristics into context-rich prompts, the LLM produces high-level driving strategies through chain-of-thought reasoning. A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness across diverse driving conditions. The experimental results, evaluated across multiple traffic scenarios, show that TeLL-Drive outperforms existing baseline methods, including other LLM-based approaches, in terms of success rates, average returns, and real-time feasibility. Ablation studies underscore the importance of each model component, especially the synergy between the attention mechanism and LLM-driven guidance. Finally, we build a virtual-real fusion experimental platform to verify the real-time performance, robustness, and reliability of the algorithm running on real vehicles through vehicle-in-loop experiments.

📄 PDF Abstract BibTeX arXiv:2502.01387

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous DrivingDecision MakingDeep Reinforcement Learning

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving

2026-01-29 · Weitong Lian, Zecong Tang, Haoran Li, Tianjian Gao 외 arxiv

Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain. However, large models demand substantial GPU memory a…

Knowledge DistillationAutonomous Driving

Towards Knowledge-driven Autonomous Driving

2023-12-07 · Xin Li, Yeqi Bai, Pinlong Cai, Licheng Wen 외

This paper explores the emerging knowledge-driven autonomous driving technologies. Our investigation highlights the limitations of current autonomous driving systems, in particular their sensitivity to data bias, difficu…

Autonomous DrivingNeural Rendering

DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking

2025-07-28 · Weicheng Zheng, Xiaofei Mao, Nanfei Ye, Pengxiang Li 외 arxiv

The advent of Vision-Language Models (VLMs) has significantly advanced end-to-end autonomous driving, demonstrating powerful reasoning abilities for high-level behavior planning tasks. However, existing methods are often…

Reinforcement LearningAutonomous DrivingVisual Reasoning

VLM-Assisted Continual learning for Visual Question Answering in Self-Driving

2025-02-02 · Yuxin Lin, Mengshi Qi, Liang Liu, Huadong Ma

In this paper, we propose a novel approach for solving the Visual Question Answering (VQA) task in autonomous driving by integrating Vision-Language Models (VLMs) with continual learning. In autonomous driving, VQA plays…

Autonomous DrivingContinual LearningKnowledge DistillationQuestion Answering+2

DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving

2024-09-16 · Songning Lai, Tianlang Xue, Hongru Xiao, Lijie Hu 외

Recent advancements in autonomous driving have seen a paradigm shift towards end-to-end learning paradigms, which map sensory inputs directly to driving actions, thereby enhancing the robustness and adaptability of auton…

Autonomous DrivingAutonomous Vehicles