Joint multi-modal Self-Supervised pre-training in Remote Sensing: Application to Methane Source Classification
With the current ubiquity of deep learning methods to solve computer vision and remote sensing specific tasks, the need for labelled data is growing constantly. However, in many cases, the annotation process can be long and tedious depending on the expertise needed to perform reliable annotations. In order to alleviate this need for annotations, several self-supervised methods have recently been proposed in the literature. The core principle behind these methods is to learn an image encoder using solely unlabelled data samples. In earth observation, there are opportunities to exploit domain-specific remote sensing image data in order to improve these methods. Specifically, by leveraging the geographical position associated with each image, it is possible to cross reference a location captured from multiple sensors, leading to multiple views of the same locations. In this paper, we briefly review the core principles behind so-called joint-embeddings methods and investigate the usage of multiple remote sensing modalities in self-supervised pre-training. We evaluate the final performance of the resulting encoders on the task of methane source classification.
Code (0)
등록된 구현이 없습니다.
Tasks
Earth ObservationSimilar Papers 제목 키워드 기반
Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning
Self-supervised learning is an efficient pre-training method for medical image analysis. However, current research is mostly confined to specific-modality data pre-training, consuming considerable time and resources with…
Continual LearningContinual Self-Supervised LearningMedical Image AnalysisRepresentation Learning+1Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training
Masked Autoencoders (MAE) have shown promising performance in self-supervised learning for both 2D and 3D computer vision. However, existing MAE-style methods can only learn from the data of a single modality, i.e., eith…
DecoderPoint Cloud Pre-trainingRepresentation LearningSelf-Supervised LearningSelf-supervised learning of imaging and clinical signatures using a multimodal joint-embedding predictive architecture
The development of multimodal models for pulmonary nodule diagnosis is limited by the scarcity of labeled data and the tendency for these models to overfit on the training distribution. In this work, we leverage self-sup…
Self-Supervised LearningTVDIM: Enhancing Image Self-Supervised Pretraining via Noisy Text Data
Among ubiquitous multimodal data in the real world, text is the modality generated by human, while image reflects the physical world honestly. In a visual understanding application, machines are expected to understand im…
Contrastive Learningimage-classificationImage ClassificationSelf-Supervised LearningSelf-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes
Recent neuroimaging studies that focus on predicting brain disorders via modern machine learning approaches commonly include a single modality and rely on supervised over-parameterized models.However, a single modality p…
DiagnosticSelf-Supervised Learning