paper-with-me

홈 › Papers

METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidance

2024-09-19 · Ziang Guo, Xinhao Lin, Zakhar Yagudin, Artem Lykov, Yong Wang, Yanqiang Li, Dzmitry Tsetserukou

Multi-modal end-to-end autonomous driving has shown promising advancements in recent work. By embedding more modalities into end-to-end networks, the system's understanding of both static and dynamic aspects of the driving environment is enhanced, thereby improving the safety of autonomous driving. In this paper, we introduce METDrive, an end-to-end system that leverages temporal guidance from the embedded time series features of ego states, including rotation angles, steering, throttle signals, and waypoint vectors. The geometric features derived from perception sensor data and the time series features of ego state data jointly guide the waypoint prediction with the proposed temporal guidance loss function. We evaluated METDrive on the CARLA leaderboard benchmarks, achieving a driving score of 70%, a route completion score of 94%, and an infraction score of 0.78.

📄 PDF Abstract BibTeX arXiv:2409.12667

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous DrivingTime Series

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음
PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…
CARLA CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban…

Similar Papers 제목 키워드 기반

H-MBA: Hierarchical MamBa Adaptation for Multi-Modal Video Understanding in Autonomous Driving

2025-01-08 · Siran Chen, Yuxiao Luo, Yue Ma, Yu Qiao 외

With the prevalence of Multimodal Large Language Models(MLLMs), autonomous driving has encountered new opportunities and challenges. In particular, multi-modal video understanding is critical to interactively analyze wha…

Autonomous DrivingMambaobject-detectionObject Detection+2

UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs

2025-11-03 · Zhe Liu, Jinghua Hou, Xiaoqing Ye, Jingdong Wang 외 arxiv

Although transformers have demonstrated remarkable capabilities across various domains, their quadratic attention mechanisms introduce significant computational overhead when processing long-sequence data. In this paper,…

3D Object DetectionTemporal Sequences3D Object TrackingAutonomous Driving

A Framework for a Capability-driven Evaluation of Scenario Understanding for Multimodal Large Language Models in Autonomous Driving

2025-03-14 · Tin Stribor Sohn, Philipp Reis, Maximilian Dillitzer, Johannes Bach 외

Multimodal large language models (MLLMs) hold the potential to enhance autonomous driving by combining domain-independent world knowledge with context-specific language guidance. Their integration into autonomous driving…

Autonomous DrivingDecision MakingWorld Knowledge

PRAM-R: A Perception-Reasoning-Action-Memory Framework with LLM-Guided Modality Routing for Adaptive Autonomous Driving

2026-03-04 · Yi Zhang, Xian Zhang, Saisi Zhao, Yinglei Song 외 arxiv

Multimodal perception enables robust autonomous driving but incurs unnecessary computational cost when all sensors remain active. This paper presents PRAM-R, a unified Perception-Reasoning-Action-Memory framework with LL…

Autonomous Driving

AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving

2024-05-13 · Daniel Bogdoll, Iramm Hamdard, Lukas Namgyu Rößler, Felix Geisler 외

The scale-up of autonomous vehicles depends heavily on their ability to deal with anomalies, such as rare objects on the road. In order to handle such situations, it is necessary to detect anomalies in the first place. A…

Anomaly DetectionAutonomous DrivingAutonomous Vehicles