Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving
Recent breakthroughs in large language models (LLMs) have not only advanced natural language processing but also inspired their application in domains with structurally similar problems--most notably, autonomous driving motion generation. Both domains involve autoregressive sequence modeling, token-based representations, and context-aware decision making, making the transfer of LLM components a natural and increasingly common practice. However, despite promising early attempts, a systematic understanding of which LLM modules are truly transferable remains lacking. In this paper, we present a comprehensive evaluation of five key LLM modules--tokenizer design, positional embedding, pre-training paradigms, post-training strategies, and test-time computation--within the context of motion generation for autonomous driving. Through extensive experiments on the Waymo Sim Agents benchmark, we demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation. In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios. We evaluate our method on the Sim Agents task and achieve competitive results.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingDecision MakingSimilar Papers 제목 키워드 기반
DRL-Based Trajectory Tracking for Motion-Related Modules in Autonomous Driving
Autonomous driving systems are always built on motion-related modules such as the planner and the controller. An accurate and robust trajectory tracking method is indispensable for these motion-related modules as a primi…
Autonomous DrivingDeep Reinforcement LearningRepresentation LearningTowards Safe Autonomous Driving: A Real-Time Motion Planning Algorithm on Embedded Hardware
Ensuring the functional safety of Autonomous Vehicles (AVs) requires motion planning modules that not only operate within strict real-time constraints but also maintain controllability in case of system faults. Existing …
Trajectory PlanningAutonomous VehiclesAutonomous DrivingMotion PlanningAutonomous LLM-Enhanced Adversarial Attack for Text-to-Motion
Human motion generation driven by deep generative models has enabled compelling applications, but the ability of text-to-motion (T2M) models to produce realistic motions from text prompts raises security concerns if expl…
Adversarial AttackAdversarial TextMotion GenerationMyGo: Consistent and Controllable Multi-View Driving Video Generation with Camera Control
High-quality driving video generation is crucial for providing training data for autonomous driving models. However, current generative models rarely focus on enhancing camera motion control under multi-view tasks, which…
Autonomous DrivingVideo GenerationSitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes
Recent advancements in human motion synthesis have focused on specific types of motions, such as human-scene interaction, locomotion or human-human interaction, however, there is a lack of a unified system capable of gen…
Motion GenerationMotion SynthesisPose Retrieval