Dual-former: Hybrid Self-attention Transformer for Efficient Image Restoration
Recently, image restoration transformers have achieved comparable performance with previous state-of-the-art CNNs. However, how to efficiently leverage such architectures remains an open problem. In this work, we present Dual-former whose critical insight is to combine the powerful global modeling ability of self-attention modules and the local modeling ability of convolutions in an overall architecture. With convolution-based Local Feature Extraction modules equipped in the encoder and the decoder, we only adopt a novel Hybrid Transformer Block in the latent layer to model the long-distance dependence in spatial dimensions and handle the uneven distribution between channels. Such a design eliminates the substantial computational complexity in previous image restoration transformers and achieves superior performance on multiple image restoration tasks. Experiments demonstrate that Dual-former achieves a 1.91dB gain over the state-of-the-art MAXIM method on the Indoor dataset for single image dehazing while consuming only 4.2% GFLOPs as MAXIM. For single image deraining, it exceeds the SOTA method by 0.1dB PSNR on the average results of five datasets with only 21.5% GFLOPs. Dual-former also substantially surpasses the latest desnowing method on various datasets, with fewer parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderImage DehazingImage RestorationRain RemovalSingle Image DehazingSingle Image DerainingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HAAT: Hybrid Attention Aggregation Transformer for Image Super-Resolution
In the research area of image super-resolution, Swin-transformer-based models are favored for their global spatial modeling and shifting window attention mechanism. However, existing methods often limit self-attention to…
Image Super-ResolutionSuper-ResolutionA survey of the Vision Transformers and their CNN-Transformer based Variants
Vision transformers have become popular as a possible substitute to convolutional neural networks (CNNs) for a variety of computer vision applications. These transformers, with their ability to focus on global relationsh…
SurveyAccelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network
Due to the highly parallelizable architecture, Transformer is faster to train than RNN-based models and popularly used in machine translation tasks. However, at inference time, each output word requires all the hidden st…
DecoderKnowledge DistillationMachine TranslationTranslationDIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation
Although purely transformer-based architectures showed promising performance in many computer vision tasks, many hybrid models consisting of CNN and transformer blocks are introduced to fit more specialized tasks. Nevert…
Image SegmentationMedical Image SegmentationSemantic SegmentationIntegrating Quantum-Classical Attention in Patch Transformers for Enhanced Time Series Forecasting
QCAAPatchTF is a quantum attention network integrated with an advanced patch-based transformer, designed for multivariate time series forecasting, classification, and anomaly detection. Leveraging quantum superpositions,…
Anomaly DetectionMultivariate Time Series ForecastingTime SeriesTime Series Forecasting