Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml
In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx ZCU102 evaluation board. The latency is reduced to 3 ms per image when increasing the batch size to ten, corresponding to the use case where the autonomous vehicle receives inputs from multiple cameras simultaneously. We show, through aggressive filter reduction and heterogeneous quantization-aware training, and an optimized implementation of convolutional layers, that the power consumption and resource utilization can be significantly reduced while maintaining accuracy on the Cityscapes dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingAutonomous VehiclesQuantizationReal-Time Semantic SegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hardware-Enforced Semantic Coordination for Safety-Critical Real-Time Autonomous Systems
Recent advances in agentic AI are producing increasingly complex autonomous systems that integrate large language models, world models, optimization engines, specialized neural architectures, autonomous platforms, and hu…
SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud
4D LiDAR semantic segmentation, also referred to as multi-scan semantic segmentation, plays a crucial role in enhancing the environmental understanding capabilities of autonomous vehicles or robots. It classifies the sem…
Autonomous DrivingAutonomous NavigationAutonomous VehiclesLIDAR Semantic Segmentation+2FPGA based Parallelized Architecture of Efficient Graph based Image Segmentation Algorithm
Efficient and real time segmentation of color images has a variety of importance in many fields of computer vision such as image compression, medical imaging, mapping and autonomous navigation. Being one of the most comp…
Autonomous NavigationImage CompressionImage SegmentationSegmentation+1PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network
The autonomous car must recognize the driving environment quickly for safe driving. As the Light Detection And Range (LiDAR) sensor is widely used in the autonomous car, fast semantic segmentation of LiDAR point cloud, w…
3D Semantic SegmentationSegmentationSemantic SegmentationReal-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images
Semantic segmentation has made striking progress due to the success of deep convolutional neural networks. Considering the demands of autonomous driving, real-time semantic segmentation has become a research hotspot thes…
Autonomous DrivingAutonomous VehiclesReal-Time Semantic SegmentationSegmentation+1