DGMR: Diffusion Guided Masked Reconstruction Framework for Multimodal Cloud Removal
Cloudy conditions affect the quality of captured data by optical satellites. Multimodal techniques rely on synthetic aperture radar (SAR) images to recover cloudy pixels in optical images. These techniques face challenges of noise, modality, and temporal differences. In this work, we propose a diffusion guided masked reconstruction (DGMR) framework for multimodal cloud removal, which consists of a masked reconstruction network (MRNet), conditional diffusion guidance model (CDGM), and noncloudy difference similarity (NDS) soft constraint. DGMR effectively extracts local-global relationships and combines complementary information using MRNet with coupled feature fusion and decoupled masked reconstruction. CDGM guides the intermediate features of MRNet to reconstruct more refined, cloud-free images. NDS ensures that the reconstructed output is consistent with temporal changes. DGMR achieves state-of-the-art results on four widely used benchmarks of the SEN12MS-CR, M3R-CR, and SMILE-CR datasets. The code and trained models are available at https://github.com/chouhan-avinash/DGMR/
Code (1)
Tasks
Cloud RemovalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification
Diffusion models have been widely adopted in image generation, producing higher-quality and more diverse samples than generative adversarial networks (GANs). We introduce a latent diffusion model (LDM) for precipitation …
Image GenerationUncertainty QuantificationDisentangling and Generating Modalities for Recommendation in Missing Modality Scenarios
Multi-modal recommender systems (MRSs) have achieved notable success in improving personalization by leveraging diverse modalities such as images, text, and audio. However, two key challenges remain insufficiently addres…
Cross-Modal RetrievalRecommendation SystemsAdaptive Mask-guided K-space Diffusion for Accelerated MRI Reconstruction
As the deep learning revolution marches on, masked modeling has emerged as a distinctive approach that involves predicting parts of the original data that are proportionally masked during training, and has demonstrated e…
MRI ReconstructionSecDiff: Diffusion-Aided Secure Deep Joint Source-Channel Coding Against Adversarial Attacks
Deep joint source-channel coding (JSCC) has emerged as a promising paradigm for semantic communication, delivering significant performance gains over conventional separate coding schemes. However, existing JSCC framework…
Semantic CommunicationDiffusion-Guided Pretraining for Brain Graph Foundation Models
With the growing interest in foundation models for brain signals, graph-based pretraining has emerged as a promising paradigm for learning transferable representations from connectome data. However, existing contrastive …