Semantic-guided Adversarial Diffusion Model for Self-supervised Shadow Removal
Existing unsupervised methods have addressed the challenges of inconsistent paired data and tedious acquisition of ground-truth labels in shadow removal tasks. However, GAN-based training often faces issues such as mode collapse and unstable optimization. Furthermore, due to the complex mapping between shadow and shadow-free domains, merely relying on adversarial learning is not enough to capture the underlying relationship between two domains, resulting in low quality of the generated images. To address these problems, we propose a semantic-guided adversarial diffusion framework for self-supervised shadow removal, which consists of two stages. At first stage a semantic-guided generative adversarial network (SG-GAN) is proposed to carry out a coarse result and construct paired synthetic data through a cycle-consistent structure. Then the coarse result is refined with a diffusion-based restoration module (DBRM) to enhance the texture details and edge artifact at second stage. Meanwhile, we propose a multi-modal semantic prompter (MSP) that aids in extracting accurate semantic information from real images and text, guiding the shadow removal network to restore images better in SG-GAN. We conduct experiments on multiple public datasets, and the experimental results demonstrate the effectiveness of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkShadow RemovalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge
Adversarial diffusion and diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations. Adversarial approaches require target-domain adversarial loss during training, whic…
Image-to-Image TranslationDiffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation
Vessel segmentation in medical images is one of the important tasks in the diagnosis of vascular diseases and therapy planning. Although learning-based segmentation approaches have been extensively studied, a large amoun…
DenoisingRepresentation LearningSegmentationSemi-supervised Semantics-guided Adversarial Training for Trajectory Prediction
Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small c…
Adversarial RobustnessDecision MakingPredictionTrajectory PredictionSelf-Guided Diffusion Models
Diffusion models have demonstrated remarkable progress in image generation quality, especially when guidance is used to control the generative process. However, guidance requires a large amount of image-annotation pairs …
Image GenerationSemi-supervised Semantics-guided Adversarial Training for Robust Trajectory Prediction
Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where sma…
Decision MakingPredictionTrajectory Prediction