Multimodal End-to-End Learning for Autonomous Steering in Adverse Road and Weather Conditions
Autonomous driving is challenging in adverse road and weather conditions in which there might not be lane lines, the road might be covered in snow and the visibility might be poor. We extend the previous work on end-to-end learning for autonomous steering to operate in these adverse real-life conditions with multimodal data. We collected 28 hours of driving data in several road and weather conditions and trained convolutional neural networks to predict the car steering wheel angle from front-facing color camera images and lidar range and reflectance data. We compared the CNN model performances based on the different modalities and our results show that the lidar modality improves the performances of different multimodal sensor-fusion models. We also performed on-road tests with different models and they support this observation.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingSensor FusionSimilar Papers 제목 키워드 기반
Leveraging Road Area Semantic Segmentation with Auxiliary Steering Task
Robustness of different pattern recognition methods is one of the key challenges in autonomous driving, especially when driving in the high variety of road environments and weather conditions, such as gravel roads and sn…
Autonomous DrivingSegmentationSemantic SegmentationTransfer Learning3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds
Robust point cloud parsing under all-weather conditions is crucial to level-5 autonomy in autonomous driving. However, how to learn a universal 3D semantic segmentation (3DSS) model is largely neglected as most existing …
3D Semantic SegmentationAutonomous DrivingSemantic SegmentationSAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather
Multimodal sensor fusion is an essential capability for autonomous robots, enabling object detection and decision-making in the presence of failing or uncertain inputs. While recent fusion methods excel in normal environ…
3D Object DetectionAutonomous VehiclesAutonomous DrivingReal-Time Environment Condition Classification for Autonomous Vehicles
Current autonomous driving technologies are being rolled out in geo-fenced areas with well-defined operation conditions such as time of operation, area, weather conditions and road conditions. In this way, challenging co…
Autonomous DrivingAutonomous VehiclesClassificationUAV-based Intelligent Information Systems on Winter Road Safety for Autonomous Vehicles
As autonomous vehicles continue to revolutionize transportation, addressing challenges posed by adverse weather conditions, particularly during winter, becomes paramount for ensuring safe and efficient operations. One of…
Autonomous VehiclesDecision MakingSelf-Driving Cars