Multimodal Image Super-resolution via Deep Unfolding with Side Information
Deep learning methods have been successfully applied to various computer vision tasks. However, existing neural network architectures do not per se incorporate domain knowledge about the addressed problem, thus, understanding what the model has learned is an open research topic. In this paper, we rely on the unfolding of an iterative algorithm for sparse approximation with side information, and design a deep learning architecture for multimodal image super-resolution that incorporates sparse priors and effectively utilizes information from another image modality. We develop two deep models performing reconstruction of a high-resolution image of a target image modality from its low-resolution variant with the aid of a high-resolution image from a second modality. We apply the proposed models to super-resolve near-infrared images using as side information high-resolution RGB\ images. Experimental results demonstrate the superior performance of the proposed models against state-of-the-art methods including unimodal and multimodal approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Image Super-ResolutionSuper-ResolutionSimilar Papers 제목 키워드 기반
Multimodal Deep Unfolding for Guided Image Super-Resolution
The reconstruction of a high resolution image given a low resolution observation is an ill-posed inverse problem in imaging. Deep learning methods rely on training data to learn an end-to-end mapping from a low-resolutio…
Image Super-ResolutionMultimodal Deep LearningSuper-ResolutionUnfolded Deep Kernel Estimation for Blind Image Super-resolution
Blind image super-resolution (BISR) aims to reconstruct a high-resolution image from its low-resolution counterpart degraded by unknown blur kernel and noise. Many deep neural network based methods have been proposed to …
Image Super-ResolutionSuper-ResolutionPhotothermal-SR-Net: A Customized Deep Unfolding Neural Network for Photothermal Super Resolution Imaging
This paper presents deep unfolding neural networks to handle inverse problems in photothermal radiometry enabling super resolution (SR) imaging. Photothermal imaging is a well-known technique in active thermography for n…
Super-ResolutionDeep Coupled-Representation Learning for Sparse Linear Inverse Problems with Side Information
In linear inverse problems, the goal is to recover a target signal from undersampled, incomplete or noisy linear measurements. Typically, the recovery relies on complex numerical optimization methods; recent approaches p…
Multimodal Deep LearningRepresentation LearningDeep Unfolding Network for Image Super-Resolution
Learning-based single image super-resolution (SISR) methods are continuously showing superior effectiveness and efficiency over traditional model-based methods, largely due to the end-to-end training. However, different …
Image Super-ResolutionSuper-Resolution