Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation
In the training of deep learning models, how the model parameters are initialized greatly affects the model performance, sample efficiency, and convergence speed. Representation learning for model initialization has recently been actively studied in the remote sensing field. In particular, the appearance characteristics of the imagery obtained using the a synthetic aperture radar (SAR) sensor are quite different from those of general electro-optical (EO) images, and thus representation learning is even more important in remote sensing domain. Motivated from contrastive multiview coding, we propose multi-modal representation learning for SAR semantic segmentation. Unlike previous studies, our method jointly uses EO imagery, SAR imagery, and a label mask. Several experiments show that our approach is superior to the existing methods in model performance, sample efficiency, and convergence speed.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSemantic SegmentationSimilar Papers 제목 키워드 기반
What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment
Zero-shot visual decoding from electroencephalography (EEG) aims to infer visual semantics from non-invasive neural recordings, but remains challenging due to the low signal-to-noise ratio, non-stationarity, and limited …
Representation LearningContrastive LearningGraph LearningHierarchical Consensus Network for Multiview Feature Learning
Multiview feature learning aims to learn discriminative features by integrating the distinct information in each view. However, most existing methods still face significant challenges in learning view-consistency feature…
Contrastive LearningMultiview LearningContrastive Multiview Coding
Humans view the world through many sensory channels, e.g., the long-wavelength light channel, viewed by the left eye, or the high-frequency vibrations channel, heard by the right ear. Each view is noisy and incomplete, b…
Contrastive LearningSelf-Supervised Action RecognitionSelf-Supervised Image ClassificationUnsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognition
In this paper, we introduce MultiviewVLM, a vision-language model designed for unsupervised contrastive multiview representation learning of facial emotions from 3D/4D data. Our architecture integrates pseudo-labels deri…
Contrastive LearningFacial Expression RecognitionLanguage ModelingLanguage Modelling+1Deep Contrastive Multiview Network Embedding
Multiview network embedding aims at projecting nodes in the network to low-dimensional vectors, while preserving their multiple relations and attribute information. Contrastive learning approaches have shown promising pe…
AttributeContrastive LearningGraph EmbeddingNetwork Embedding