HawkDrive: A Transformer-driven Visual Perception System for Autonomous Driving in Night Scene
Many established vision perception systems for autonomous driving scenarios ignore the influence of light conditions, one of the key elements for driving safety. To address this problem, we present HawkDrive, a novel perception system with hardware and software solutions. Hardware that utilizes stereo vision perception, which has been demonstrated to be a more reliable way of estimating depth information than monocular vision, is partnered with the edge computing device Nvidia Jetson Xavier AGX. Our software for low light enhancement, depth estimation, and semantic segmentation tasks, is a transformer-based neural network. Our software stack, which enables fast inference and noise reduction, is packaged into system modules in Robot Operating System 2 (ROS2). Our experimental results have shown that the proposed end-to-end system is effective in improving the depth estimation and semantic segmentation performance. Our dataset and codes will be released at https://github.com/ZionGo6/HawkDrive.
Code (1)
Tasks
Autonomous DrivingDepth EstimationEdge-computingSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
GPTR: Gestalt-Perception Transformer for Diagram Object Detection
Diagram object detection is the key basis of practical applications such as textbook question answering. Because the diagram mainly consists of simple lines and color blocks, its visual features are sparser than those of…
DecoderObjectobject-detectionObject Detection+1Evaluating Graphical Perception Capabilities of Vision Transformers
Vision Transformers, ViTs, have emerged as a powerful alternative to convolutional neural networks, CNNs, in a variety of image-based tasks. While CNNs have previously been evaluated for their ability to perform graphica…
Perceptual Reality Transformer: Neural Architectures for Simulating Neurological Perception Conditions
Neurological conditions affecting visual perception create profound experiential divides between affected individuals and their caregivers, families, and medical professionals. We present the Perceptual Reality Transform…
VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI
Spiking neural networks (SNNs) are brain-inspired, event-driven models that compute with sparse spikes, which enables highly efficient visual perception in resource-constrained embodied AI models. The emergence of Spikin…
Visual Place RecognitionKnowledge DistillationVisual Prompting for Robotic Manipulation with Annotation-Guided Pick-and-Place Using ACT
Robotic pick-and-place tasks in convenience stores pose challenges due to dense object arrangements, occlusions, and variations in object properties such as color, shape, size, and texture. These factors complicate traje…
Trajectory Planning