ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning

2024-09-13 · Pei Deng, Wenqian Zhou, Hanlin Wu

Remote sensing (RS) change analysis is vital for monitoring Earth's dynamic processes by detecting alterations in images over time. Traditional change detection excels at identifying pixel-level changes but lacks the ability to contextualize these alterations. While recent advancements in change captioning offer natural language descriptions of changes, they do not support interactive, user-specific queries. To address these limitations, we introduce ChangeChat, the first bitemporal vision-language model (VLM) designed specifically for RS change analysis. ChangeChat utilizes multimodal instruction tuning, allowing it to handle complex queries such as change captioning, category-specific quantification, and change localization. To enhance the model's performance, we developed the ChangeChat-87k dataset, which was generated using a combination of rule-based methods and GPT-assisted techniques. Experiments show that ChangeChat offers a comprehensive, interactive solution for RS change analysis, achieving performance comparable to or even better than state-of-the-art (SOTA) methods on specific tasks, and significantly surpassing the latest general-domain model, GPT-4. Code and pre-trained weights are available at https://github.com/hanlinwu/ChangeChat.

📄 PDF Abstract BibTeX arXiv:2409.08582

Code (1)

hanlinwu/changechat 공식 구현 pytorch

Tasks

Change DetectionLanguage ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

DeltaVLM: Interactive Remote Sensing Image Change Analysis via Instruction-guided Difference Perception

2025-07-30 · Pei Deng, Wenqian Zhou, Hanlin Wu arxiv

Accurate interpretation of land-cover changes in multi-temporal satellite imagery is critical for real-world scenarios. However, existing methods typically provide only one-shot change masks or static captions, limiting …

Visual Question Answering Change Detection

Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis

2024-03-28 · Chenyang Liu, Keyan Chen, Haotian Zhang, Zipeng Qi 외

Monitoring changes in the Earth's surface is crucial for understanding natural processes and human impacts, necessitating precise and comprehensive interpretation methodologies. Remote sensing satellite imagery offers a …

Change Detection Language Modelling Large Language Model Object Counting

Effective Features of Remote Sensing Image Classification Using Interactive Adaptive Thresholding Method

2014-01-30 · T. Balaji, Dr. M. Sumathi

Remote sensing image classification can be performed in many different ways to extract meaningful features. One common approach is to perform edge detection. A second approach is to try and detect whole shapes, given the…

Classification Combinatorial Optimization Edge Detection feature selection+4

Deep learning-based interactive segmentation in remote sensing

2023-08-25 · Zhe Wang, Shoukun Sun, Xiang Que, Xiaogang Ma 외

Interactive segmentation, a computer vision technique where a user provides guidance to help an algorithm segment a feature of interest in an image, has achieved outstanding accuracy and efficient human-computer interact…

Deep Learning Interactive Segmentation Land Cover Classification Segmentation+1

Towards Comprehensive Interactive Change Understanding in Remote Sensing: A Large-scale Dataset and Dual-granularity Enhanced VLM

2025-09-27 · Junxiao Xue, Quan Deng, Xuecheng Wu, Kelu Yao 외 arxiv

Remote sensing change understanding (RSCU) is essential for analyzing remote sensing images and understanding how human activities affect the environment. However, existing datasets lack deep understanding and interactio…

Semantic Similarity