ChangeCLIP: Remote sensing change detection with multimodal vision-language representation learning
Remote sensing change detection (RSCD), which aims to identify surface changes from bitemporal images, is significant for many applications, such as environmental protection and disaster monitoring. In the last decade, driven by the wave of artificial intelligence, many change detection methods based on deep learning emerged and have achieved essential breakthroughs. However, these methods pay more attention to visual representation learning while ignoring the potential of multimodal data. Recently, the foundation vision-language model, i.e. CLIP, has provided a new paradigm for multimodal AI, demonstrating impressive performance on downstream tasks. Following this trend, in this study, we introduce ChangeCLIP, a novel framework that leverages robust semantic information from image-text pairs, specifically tailored for Remote Sensing Change Detection (RSCD). Specifically, we reconstruct the original CLIP to extract bitemporal features and propose a novel differential features compensation module to capture the detailed semantic changes between them. Besides, we proposed a vision-language-driven decoder by combining the results of image-text encoding with the visual features of the decoding stage, thereby enhancing the image semantics. The proposed ChangeCLIP achieved state-of-the-art IoU on 5 well-known change detection datasets, LEVIR-CD (85.20%), LEVIR-CD+ (75.63%), WHUCD (90.15%), CDD (95.87%) and SYSU-CD (71.41%). The code and the pretrained models of ChangeCLIP will be publicly available on https://github.com/dyzy41/ChangeCLIP.
Code (1)
Tasks
Change DetectionDecoderLanguage ModellingRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Single-temporal Supervised Remote Change Detection for Domain Generalization
Change detection is widely applied in remote sensing image analysis. Existing methods require training models separately for each dataset, which leads to poor domain generalization. Moreover, these methods rely heavily o…
Change DetectionContrastive LearningDomain GeneralizationPrompt LearningOmniCD: A Foundational Framework for Remote Sensing Image Change Detection Guided by Multimodal Semantics
Change detection (CD) in remote sensing is vital for applications such as urban monitoring and disaster assessment, yet traditional methods struggle with generalization across diverse scenarios. We present OmniCD, a foun…
Change DetectionReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining
Remote sensing image change detection is one of the fundamental tasks in remote sensing intelligent interpretation. Its core objective is to identify changes within change regions of interest (CRoI). Current multimodal l…
Multimodal ReasoningChange DetectionSupervising Remote Sensing Change Detection Models with 3D Surface Semantics
Remote sensing change detection, identifying changes between scenes of the same location, is an active area of research with a broad range of applications. Recent advances in multimodal self-supervised pretraining have r…
Change DetectionRepresentation LearningChange Detection between Multimodal Remote Sensing Data Using Siamese CNN
Detecting topographic changes in the urban environment has always been an important task for urban planning and monitoring. In practice, remote sensing data are often available in different modalities and at different ti…
Change Detection