Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning
Unsupervised multimodal change detection is a practical and challenging topic that can play an important role in time-sensitive emergency applications. To address the challenge that multimodal remote sensing images cannot be directly compared due to their modal heterogeneity, we take advantage of two types of modality-independent structural relationships in multimodal images. In particular, we present a structural relationship graph representation learning framework for measuring the similarity of the two structural relationships. Firstly, structural graphs are generated from preprocessed multimodal image pairs by means of an object-based image analysis approach. Then, a structural relationship graph convolutional autoencoder (SR-GCAE) is proposed to learn robust and representative features from graphs. Two loss functions aiming at reconstructing vertex information and edge information are presented to make the learned representations applicable for structural relationship similarity measurement. Subsequently, the similarity levels of two structural relationships are calculated from learned graph representations and two difference images are generated based on the similarity levels. After obtaining the difference images, an adaptive fusion strategy is presented to fuse the two difference images. Finally, a morphological filtering-based postprocessing approach is employed to refine the detection results. Experimental results on five datasets with different modal combinations demonstrate the effectiveness of the proposed method.
Code (1)
Tasks
Change DetectionGraph Representation LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Change Detection Between Optical Remote Sensing Imagery and Map Data via Segment Anything Model (SAM)
Unsupervised multimodal change detection is pivotal for time-sensitive tasks and comprehensive multi-temporal Earth monitoring. In this study, we explore unsupervised multimodal change detection between two key remote se…
Change DetectionSegmentationDeep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection
Image translation with convolutional neural networks has recently been used as an approach to multimodal change detection. Existing approaches train the networks by exploiting supervised information of the change areas, …
Change DetectionTranslationMultimodal Task Representation Memory Bank vs. Catastrophic Forgetting in Anomaly Detection
Unsupervised Continuous Anomaly Detection (UCAD) faces significant challenges in multi-task representation learning, with existing methods suffering from incomplete representation and catastrophic forgetting. Unlike supe…
Anomaly DetectionContrastive LearningRepresentation LearningPrecise Change Point Detection using Spectral Drift Detection
The notion of concept drift refers to the phenomenon that the data generating distribution changes over time; as a consequence machine learning models may become inaccurate and need adjustment. In this paper we consider …
Change Point DetectionDrift DetectionAdaptive Graph Convolution and Semantic-Guided Attention for Multimodal Risk Detection in Social Networks
This paper focuses on the detection of potentially dangerous tendencies of social media users in an innovative multimodal way. We integrate Natural Language Processing (NLP) and Graph Neural Networks (GNNs) together. Fir…
Keyword Extraction