Real-time object detection and tracking using flash LiDAR imagery
In this study, we present a real-time vehicle detection program that combines the You-Only-Look-Once-X (YOLOX) object detection algorithm with a multi-object Kalman filter tracker, specifically designed for analyzing 3D light detection and ranging (LiDAR) data. The use of an active imager, such as LiDAR, offers significant advantages over conventional passive 2D imagery. By providing its illumination source, LiDAR eliminates color fluctuations caused by shadowing or diurnal cycling, resulting in improved precision and accuracy for object detection and classification. Our approach involves capturing videos of 8 vehicles using an Advanced Scientific Concepts TigerCub 3D Flash LiDAR camera, which provides intensity and range data sequences. These sequences are then converted into representative color images, which are used to train the YOLOX object detector neural network. To further enhance the detection accuracy for obscured vehicles and minimize the wrong detection rate, we integrate Kalman filter trackers into the detection algorithm. These trackers identify the vehicles and predict their future locations, effectively reducing both false positive and false negative detections. The resulting algorithm is lightweight and capable of producing highly accurate inference results in near real-time on a live-stream of LiDAR data. To demonstrate the applicability of our approach on small, unmanned vehicles/drones, we deploy the application on NVIDIA's Jetson Orin Nano embedded processor for AI. By optimizing the code using TensorRT for real-time performance, we achieve object detection and classification of flash LiDAR data at an average precision exceeding 95% and a rate of 60 frames-per-second. MATLAB plays a crucial role in enabling rapid prototyping and algorithm testing, facilitating the smooth transfer and deployment of the complex deep learning logic to an edge device without compromising performance or accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object ClassificationObjectobject-detectionObject DetectionReal-Time Object Detectionvehicle detectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SAM2Auto: Auto Annotation Using FLASH
Vision-Language Models (VLMs) lag behind Large Language Models due to the scarcity of annotated datasets, as creating paired visual-textual annotations is labor-intensive and expensive. To address this bottleneck, we int…
Instance SegmentationObjectobject-detectionObject Detection+5aUToLights: A Robust Multi-Camera Traffic Light Detection and Tracking System
Following four successful years in the SAE AutoDrive Challenge Series I, the University of Toronto is participating in the Series II competition to develop a Level 4 autonomous passenger vehicle capable of handling vario…
Autonomous Vehiclesobject-detectionObject DetectionFlashSLAM: Accelerated RGB-D SLAM for Real-Time 3D Scene Reconstruction with Gaussian Splatting
We present FlashSLAM, a novel SLAM approach that leverages 3D Gaussian Splatting for efficient and robust 3D scene reconstruction. Existing 3DGS-based SLAM methods often fall short in sparse view settings and during larg…
3DGS3D Reconstruction3D Scene ReconstructionPoint Cloud Registration+1Stereoscopic Flash and No-Flash Photography for Shape and Albedo Recovery
We present a minimal imaging setup that harnesses both geometric and photometric approaches for shape and albedo recovery. We adopt a stereo camera and a flashlight to capture a stereo image pair and a flash/no-flash pai…
Shadow DetectionFlashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detection
Video Anomaly Detection (VAD) automatically identifies anomalous events from video, mitigating the need for human operators in large-scale surveillance deployments. However, three fundamental obstacles hinder real-world …
Anomaly DetectionGPUVideo Anomaly Detection