MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations
Fully supervised change detection methods have achieved significant advancements in performance, yet they depend severely on acquiring costly pixel-level labels. Considering that the patch-level annotations also contain abundant information corresponding to both changed and unchanged objects in bi-temporal images, an intuitive solution is to segment the changes with patch-level annotations. How to capture the semantic variations associated with the changed and unchanged regions from the patch-level annotations to obtain promising change results is the critical challenge for the weakly supervised change detection task. In this paper, we propose a memory-supported transformer (MS-Former), a novel framework consisting of a bi-directional attention block (BAB) and a patch-level supervision scheme (PSS) tailored for weakly supervised change detection with patch-level annotations. More specifically, the BAM captures contexts associated with the changed and unchanged regions from the temporal difference features to construct informative prototypes stored in the memory bank. On the other hand, the BAM extracts useful information from the prototypes as supplementary contexts to enhance the temporal difference features, thereby better distinguishing changed and unchanged regions. After that, the PSS guides the network learning valuable knowledge from the patch-level annotations, thus further elevating the performance. Experimental results on three benchmark datasets demonstrate the effectiveness of our proposed method in the change detection task. The demo code for our work will be publicly available at \url{https://github.com/guanyuezhen/MS-Former}.
Code (1)
Tasks
Change DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transformer-based Multi-Instance Learning for Weakly Supervised Object Detection
Weakly Supervised Object Detection (WSOD) enables the training of object detection models using only image-level annotations. State-of-the-art WSOD detectors commonly rely on multi-instance learning (MIL) as the backbone…
Objectobject-detectionObject DetectionWeakly Supervised Object DetectionVoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment
Vision-language pre-training (VLP) has recently proven highly effective for various uni- and multi-modal downstream applications. However, most existing end-to-end VLP methods use high-resolution image-text box data to p…
object-detectionObject DetectionReferring ExpressionReferring Expression ComprehensionJoint CNN and Transformer Network via weakly supervised Learning for efficient crowd counting
Currently, for crowd counting, the fully supervised methods via density map estimation are the mainstream research directions. However, such methods need location-level annotation of persons in an image, which is time-co…
Crowd CountingWeakly-supervised LearningT-vectors: Weakly Supervised Speaker Identification Using Hierarchical Transformer Model
Identifying multiple speakers without knowing where a speaker's voice is in a recording is a challenging task. This paper proposes a hierarchical network with transformer encoders and memory mechanism to address this pro…
Speaker IdentificationTransCrowd: weakly-supervised crowd counting with transformers
The mainstream crowd counting methods usually utilize the convolution neural network (CNN) to regress a density map, requiring point-level annotations. However, annotating each person with a point is an expensive and lab…
Crowd Counting