Towards Autonomous Driving: a Multi-Modal 360$^{\circ}$ Perception Proposal
In this paper, a multi-modal 360$^{\circ}$ framework for 3D object detection and tracking for autonomous vehicles is presented. The process is divided into four main stages. First, images are fed into a CNN network to obtain instance segmentation of the surrounding road participants. Second, LiDAR-to-image association is performed for the estimated mask proposals. Then, the isolated points of every object are processed by a PointNet ensemble to compute their corresponding 3D bounding boxes and poses. Lastly, a tracking stage based on Unscented Kalman Filter is used to track the agents along time. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A wide variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack in a real autonomous driving application.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object DetectionAutonomous DrivingAutonomous VehiclesInstance Segmentationobject-detectionObject DetectionSemantic SegmentationSensor FusionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for end-to-end autonomous driving by jointly integrating perception, reasoning, and decision making within a unified multimodal framework. However, …
Visual Question AnsweringTrajectory PlanningAutonomous DrivingDecision MakingPRAM-R: A Perception-Reasoning-Action-Memory Framework with LLM-Guided Modality Routing for Adaptive Autonomous Driving
Multimodal perception enables robust autonomous driving but incurs unnecessary computational cost when all sensors remain active. This paper presents PRAM-R, a unified Perception-Reasoning-Action-Memory framework with LL…
Autonomous DrivingV2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving
Vehicle-to-everything (V2X) communication techniques enable the collaboration between vehicles and many other entities in the neighboring environment, which could fundamentally improve the perception system for autonomou…
Autonomous DrivingOmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustworthy Vision-Language Autonomous Driving
The deployment of Vision-Language Models (VLMs) in safety-critical domains like autonomous driving (AD) is critically hindered by reliability failures, most notably object hallucination. This failure stems from their rel…
Reinforcement LearningAutonomous DrivingVisual GroundingMulti-modal Sensor Fusion for Auto Driving Perception: A Survey
Multi-modal fusion is a fundamental task for the perception of an autonomous driving system, which has recently intrigued many researchers. However, achieving a rather good performance is not an easy task due to the nois…
Autonomous Drivingobject-detectionObject DetectionSemantic Segmentation+2