Contrastive Learning of Features between Images and LiDAR
Image and Point Clouds provide different information for robots. Finding the correspondences between data from different sensors is crucial for various tasks such as localization, mapping, and navigation. Learning-based descriptors have been developed for single sensors; there is little work on cross-modal features. This work treats learning cross-modal features as a dense contrastive learning problem. We propose a Tuple-Circle loss function for cross-modality feature learning. Furthermore, to learn good features and not lose generality, we developed a variant of widely used PointNet++ architecture for point cloud and U-Net CNN architecture for images. Moreover, we conduct experiments on a real-world dataset to show the effectiveness of our loss function and network structure. We show that our models indeed learn information from both images as well as LiDAR by visualizing the features.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cross-Modal Self-Supervised Learning with Effective Contrastive Units for LiDAR Point Clouds
3D perception in LiDAR point clouds is crucial for a self-driving vehicle to properly act in 3D environment. However, manually labeling point clouds is hard and costly. There has been a growing interest in self-supervise…
3D Object Detection3D Semantic SegmentationAutonomous DrivingContrastive Learning+4ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection
In the field of 3D object detection tasks, fusing heterogeneous features from LiDAR and camera sensors into a unified Bird's Eye View (BEV) representation is a widely adopted paradigm. However, existing methods are often…
3D Object DetectionContrastive LearningDepth EstimationGraph Matching+2Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models
Contrastive image-to-LiDAR knowledge transfer, commonly used for learning 3D representations with synchronized images and point clouds, often faces a self-conflict dilemma. This issue arises as contrastive losses uninten…
Representation LearningTransfer LearningLiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes
LiDAR data pretraining offers a promising approach to leveraging large-scale, readily available datasets for enhanced data utilization. However, existing methods predominantly focus on sparse voxel representation, overlo…
Mixture-of-ExpertsRepresentation LearningMonocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching
Achieving monocular camera localization within pre-built LiDAR maps can bypass the simultaneous mapping process of visual SLAM systems, potentially reducing the computational overhead of autonomous localization. To this …
Camera LocalizationContrastive LearningCross-modal place recognitionVisual Place Recognition