M2GAN: A Multi-Stage Self-Attention Network for Image Rain Removal on Autonomous Vehicles
Image deraining is a new challenging problem in applications of autonomous vehicles. In a bad weather condition of heavy rainfall, raindrops, mainly hitting the vehicle's windshield, can significantly reduce observation ability even though the windshield wipers might be able to remove part of it. Moreover, rain flows spreading over the windshield can yield the physical effect of refraction, which seriously impede the sightline or undermine the machine learning system equipped in the vehicle. In this paper, we propose a new multi-stage multi-task recurrent generative adversarial network (M2GAN) to deal with challenging problems of raindrops hitting the car's windshield. This method is also applicable for removing raindrops appearing on a glass window or lens. M2GAN is a multi-stage multi-task generative adversarial network that can utilize prior high-level information, such as semantic segmentation, to boost deraining performance. To demonstrate M2GAN, we introduce the first real-world dataset for rain removal on autonomous vehicles. The experimental results show that our proposed method is superior to other state-of-the-art approaches of deraining raindrops in respect of quantitative metrics and visual quality. M2GAN is considered the first method to deal with challenging problems of real-world rains under unconstrained environments such as autonomous vehicles.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesGenerative Adversarial NetworkRain RemovalSemantic SegmentationSimilar Papers 제목 키워드 기반
GS-Net: Global Self-Attention Guided CNN for Multi-Stage Glaucoma Classification
Glaucoma is a common eye disease that leads to irreversible blindness unless timely detected. Hence, glaucoma detection at an early stage is of utmost importance for a better treatment plan and ultimately saving the visi…
Binary ClassificationImproved Transformer for High-Resolution GANs
Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-reso…
Image GenerationVocal Bursts Intensity PredictionMTSIC: Multi-stage Transformer-based GAN for Spectral Infrared Image Colorization
Thermal infrared (TIR) images, acquired through thermal radiation imaging, are unaffected by variations in lighting conditions and atmospheric haze. However, TIR images inherently lack color and texture information, limi…
ColorizationGenerative Adversarial NetworkImage ColorizationLocal-to-Global Self-Attention in Vision Transformers
Transformers have demonstrated great potential in computer vision tasks. To avoid dense computations of self-attentions in high-resolution visual data, some recent Transformer models adopt a hierarchical design, where se…
image-classificationImage ClassificationSemantic SegmentationSAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis
Synthesizing photo-realistic images based on text descriptions is a challenging task in the field of computer vision. Although generative adversarial networks have made significant breakthroughs in this task, they stil…
Image GenerationSemantic SimilaritySemantic Textual SimilaritySentence