Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles
Autonomous Vehicles (AVs) must make reliable decisions in dense urban environments where pedestrian behavior is variable, sometimes abnormal, and often unseen during training. Reinforcement learning (RL)-based AV control systems perform well in structured traffic but struggle to generalize to unpredictable pedestrian interactions and out-of-distribution scenarios. Their reliance on handcrafted rewards and opaque decisions further limits their suitability for safety-critical, pedestrian-rich environments. To address these limitations, we introduce a Large Language Model (LLM)-based decision-making framework for pedestrian-aware behavioral planning. The system converts structured scene observations into natural-language reasoning prompts, enabling the LLM to infer pedestrian intent, anticipate risk, and generate cautious tactical driving decisions. These decisions are executed by a motion planner that ensures smooth, kinematically feasible control. We evaluate the framework in SUMO across multiple pedestrian-interaction scenarios, including unexpected jaywalking, turn-back crossing, hesitation, and bidirectional crossing. In zero-shot evaluation, the LLM-based agent achieves a 68% collision-free success rate, substantially outperforming deep RL baselines (17.7%). With few-shot episodic memory in a single-pedestrian scenario, performance increases to 96.0%, exceeding a custom DQN controller (82.0%). Cross-behavior evaluation further shows that memory derived from turn-back interactions transfers to unseen hesitation and bidirectional crossing scenarios, achieving 82.0% and 90.0% success, respectively. The system consistently initiates earlier responses, maintains wider safety buffers, and produces interpretable, human-aligned decisions.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningAutonomous VehiclesSimilar Papers 제목 키워드 기반
PePScenes: A Novel Dataset and Baseline for Pedestrian Action Prediction in 3D
Predicting the behavior of road users, particularly pedestrians, is vital for safe motion planning in the context of autonomous driving systems. Traditionally, pedestrian behavior prediction has been realized in terms of…
Autonomous DrivingMotion PlanningPredictionTrajectory ForecastingVRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users
Advanced perception and path planning are at the core for any self-driving vehicle. Autonomous vehicles need to understand the scene and intentions of other road users for safe motion planning. For urban use cases it is …
Autonomous VehiclesMotion PlanningMulti-Task LearningUncertainty-Aware Motion Planning for Autonomous Driving in Mixed Traffic Environment
In mixed-traffic environments where autonomous and human-driven vehicles may co-exist, motion planning for autonomous vehicles requires anticipating the future behaviors of surrounding human drivers. Existing reinforceme…
Reinforcement LearningAutonomous VehiclesAutonomous DrivingMotion PlanningHarnessing ADAS for Pedestrian Safety: A Data-Driven Exploration of Fatality Reduction
Pedestrian fatalities continue to rise in the United States, driven by factors such as human distraction, increased vehicle size, and complex traffic environments. Advanced Driver Assistance Systems (ADAS) offer a promis…
On-Board Pedestrian Trajectory Prediction Using Behavioral Features
This paper presents a novel approach to pedestrian trajectory prediction for on-board camera systems, which utilizes behavioral features of pedestrians that can be inferred from visual observations. Our proposed method, …
Pedestrian Trajectory PredictionPredictionTrajectory Prediction