RGB-X Object Detection via Scene-Specific Fusion Modules
Multimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions. However, existing deep sensor fusion methods usually employ convoluted architectures with intermingled multimodal features, requiring large coregistered multimodal datasets for training. In this work, we present an efficient and modular RGB-X fusion network that can leverage and fuse pretrained single-modal models via scene-specific fusion modules, thereby enabling joint input-adaptive network architectures to be created using small, coregistered multimodal datasets. Our experiments demonstrate the superiority of our method compared to existing works on RGB-thermal and RGB-gated datasets, performing fusion using only a small amount of additional parameters. Our code is available at https://github.com/dsriaditya999/RGBXFusion.
Code (1)
Tasks
Autonomous VehiclesMultispectral Object DetectionObjectobject-detectionObject DetectionSensor FusionSimilar Papers 제목 키워드 기반
SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection
By identifying four important components of existing LiDAR-camera 3D object detection methods (LiDAR and camera candidates, transformation, and fusion outputs), we observe that all existing methods either find dense cand…
3D Object Detectionobject-detectionObject DetectionFADet: A Multi-sensor 3D Object Detection Network based on Local Featured Attention
Camera, LiDAR and radar are common perception sensors for autonomous driving tasks. Robust prediction of 3D object detection is optimally based on the fusion of these sensors. To exploit their abilities wisely remains a …
3D Object DetectionAutonomous DrivingObjectobject-detection+1MLCVNet: Multi-Level Context VoteNet for 3D Object Detection
In this paper, we address the 3D object detection task by capturing multi-level contextual information with the self-attention mechanism and multi-scale feature fusion. Most existing 3D object detection methods recognize…
3D Object DetectionObjectobject-detectionObject DetectionASY-VRNet: Waterway Panoptic Driving Perception Model based on Asymmetric Fair Fusion of Vision and 4D mmWave Radar
Panoptic Driving Perception (PDP) is critical for the autonomous navigation of Unmanned Surface Vehicles (USVs). A PDP model typically integrates multiple tasks, necessitating the simultaneous and robust execution of var…
Autonomous Navigationobject-detectionObject DetectionSegmentation+1Modular Diffusion Models for Structured Visual Recognition
Traditional supervised methods for structured visual recognition tasks -- such as object detection, segmentation, and scene graph generation -- often produce deterministic, fixed outputs, limiting their ability to captur…
Scene Graph GenerationInstance SegmentationObject Detection