Understanding Time Variations of DNN Inference in Autonomous Driving
Deep neural networks (DNNs) are widely used in autonomous driving due to their high accuracy for perception, decision, and control. In safety-critical systems like autonomous driving, executing tasks like sensing and perception in real-time is vital to the vehicle's safety, which requires the application's execution time to be predictable. However, non-negligible time variations are observed in DNN inference. Current DNN inference studies either ignore the time variation issue or rely on the scheduler to handle it. None of the current work explains the root causes of DNN inference time variations. Understanding the time variations of the DNN inference becomes a fundamental challenge in real-time scheduling for autonomous driving. In this work, we analyze the time variation in DNN inference in fine granularity from six perspectives: data, I/O, model, runtime, hardware, and end-to-end perception system. Six insights are derived in understanding the time variations for DNN inference.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingSchedulingSimilar Papers 제목 키워드 기반
RoboDriveVLM: A Novel Benchmark and Baseline towards Robust Vision-Language Models for Autonomous Driving
Current Vision-Language Model (VLM)-based end-to-end autonomous driving systems often leverage large language models to generate driving decisions directly based on their understanding of the current scene. However, such…
Knowledge DistillationTrajectory PredictionTest-time AdaptationAutonomous DrivingTraining Strategies for Vision Transformers for Object Detection
Vision-based Transformer have shown huge application in the perception module of autonomous driving in terms of predicting accurate 3D bounding boxes, owing to their strong capability in modeling long-range dependencies …
Autonomous DrivingObjectobject-detectionObject DetectionUnifying Language-Action Understanding and Generation for Autonomous Driving
Vision-Language-Action (VLA) models are emerging as a promising paradigm for end-to-end autonomous driving, valued for their potential to leverage world knowledge and reason about complex driving scenes. However, existin…
Instruction FollowingAction UnderstandingAutonomous DrivingGeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving
Vision-language models (VLMs) for autonomous driving have shown promising performance, but their ability to handle region-specific traffic rules remains underexplored, raising uncertainties about their deployment across …
Multimodal ReasoningScene UnderstandingAutonomous DrivingLogic-RAG: Augmenting Large Multimodal Models with Visual-Spatial Knowledge for Road Scene Understanding
Large multimodal models (LMMs) are increasingly integrated into autonomous driving systems for user interaction. However, their limitations in fine-grained spatial reasoning pose challenges for system interpretability an…
Autonomous DrivingRAGRetrieval-augmented Generationroad scene understanding+2